DW's blog
0%

Makefile

Posted on Edited on

The official cite is at: Manual
or use man make. The following are some templates for pasting.

Generate a sequence.

It’s useful in generate big datasets by many small files, e.g., 

1
E:=$(shell seq -f "%03g" 1 1 100)

In shell, it can be replaced by: 

1
E:=$(shell echo {1..5})

The multiple iteration are also supported,

1
E:=$(shell echo {1..5}_{1..5})

A quickly way to avoid for each can be done by

1
2
x:=$(shell echo {1..5}_{1..5}_{1..5})
grid:=$(subst _,\ ,$(x))

Get the input and output

We usually need to list filenames in a directory, they are always the input of the program, e.g., list all files with suffix .h5 

1
srl:=$(wildcard ./input/*.h5)

The output often has the similar name with only string changed, e.g.,

1
dst:=$(srl:./input/%.h5=./output/%.h5)

Target

the makefile need a target to execute, we focus on the target need iteration. For small iteration, you can 

1
2
3
4
5
6
E1:=$(shell echo {1..5})
E2:=$(shell echo {1..3})

target :=
target += $(foreach x, $(E1), $\
                $(foreach y, $(E2), $(shell echo $(x)/$(y).h5)))

It is not recommended in large numbers, foreach is much faster.

1
2
target := $(foreach x, $(E1), $\
                $(foreach y, $(E2), $(shell echo $(x)/$(y).h5)))

For complex situations, we use function to iterate, here is an template.

1
2
3
4
5
6
7
8
9
10
11
xs:=$(shell seq -f "%03g" 0 1 9)
ys:=$(shell seq -f "%03g" 0 1 9)

define rule
recon/$(1)_$(2).h5:
	mkdir -p $$(dir $$@)
	python3 em.py -o $$@ --input $(1) $(2)

endef

$(foreach x,$(xs),$(foreach y,$(ys),$(eval $(call rule,$(x),$(y)))))

you also need a target:

1
all:$(foreach x,$(xs),$(foreach y,$(ys),recon/$(x)_$(y).h5))

Notice that

  • DO NOT add blanks in the variable.
  • In template most $ need to be doubled, despite the local and global variables.

Bash style also works, but not recommanded.

1
2
3
4
5
6
7
8
all:
    @for x in $(E1);\
    do \
            for y in $(E2);\
            do \
                    echo $$x $$y; \
            done \
    done

Here is a more complex case, we need iterate multiple times (epoch), the results in each step should be recorded, we can use the following template:

1
2
3
4
5
6
7
define tpl_iter
$(guile (+ $(1) 1))/%.h5: data/type1/%.root $(1)/%.h5
    mkdir -p $$(dir $$@)
    python3 Recon_bak.py $(1) $$@ $$^
endef

$(foreach i,$(iterl),$(eval $(call tpl_iter,$(i))))

Automatic variables

variable mean
$@ target
$% target member name
$< first prerequisite
$? all prerequisites
$* match %

If we wart to specify the dependent file, use word,

1
$(word 2, $<)

If we need a calculator, use bc:

1
target:$(shell echo '$(x)+2000' | bc -l).h5

Some frequent bugs:

  • remember to create the path 
    1
    2
    3
    mkdir -p $(dir $@)
    # or
    mkdir -p $(dirname $@)
  • tab is not equal to space, be careful to the editor.