Functions | Variables
overlay_prestage_def Namespace Reference

Functions

def Subdivide (input)
 
def RecurseOverDefinitions (input, tier_num, n_tiers, overlay_def, output_def_name_base, overwrite_defns=False)
 

Variables

 GROUP_SIZE
 
 FILE_PATTERN
 
 SAM
 
 experiment
 
 parser
 
 description
 
 metavar
 
 help
 
 nargs
 
 action
 
 default
 
 args
 
 output_def_name
 
 fhicl_runs
 
 fhicl_files
 
 file_list
 
 dimensions
 
 defname
 
 n_files
 
 n_files_div10
 
 curr_perc
 
 matches
 
 md
 
 num_tiers
 
 check_output
 
 files
 
 snapid
 
 n_runs
 
 files_unmatched
 
 overlay_runs
 

Detailed Description

  overlay_prestage_def.py:
  
  Create (and snapshot) a SAM definition that contains only those files
  from an overlay definition corresponding to the files actually used
  by the overlay FCLs in the FCL definition. 

Function Documentation

def overlay_prestage_def.RecurseOverDefinitions (   input,
  tier_num,
  n_tiers,
  overlay_def,
  output_def_name_base,
  overwrite_defns = False 
)

Definition at line 24 of file overlay_prestage_def.py.

References ana.assert(), and Subdivide().

24 def RecurseOverDefinitions(input, tier_num, n_tiers, overlay_def, output_def_name_base, overwrite_defns=False):
25  if len(input) == 0:
26  return None
27 
28  group_defns = []
29  for group_idx, grouping in enumerate(input):
30 
31  # fhicl_runs (they're tuples of two values)
32  if all(len(g) == 2 and all(isinstance(i, int) for i in g) for g in grouping):
33  subset_defn_dims = "defname: %s and (%s)" % (overlay_def, " or ".join(("run_number %d.%d" % (run,sub) for run,sub in grouping)))
34  else:
35  subset_defn_dims = " or ".join("dataset_def_name_newest_snapshot %s" % g for g in grouping)
36 
37  if tier_num < n_tiers:
38  suffix = "_" + ("Sub" * (n_tiers - tier_num)) + ("set%d" % group_idx)
39  else:
40  suffix = ""
41 
42  output_def_name = output_def_name_base + suffix
43 
44  if overwrite_defns:
45  try:
46  SAM.deleteDefinition(defname=output_def_name)
48  pass
49 
50  SAM.createDefinition(defname=output_def_name, dims=subset_defn_dims)
51 
52 # print " " * (n_tiers - tier_num), "new defn:", output_def_name
53 # with open(output_def_name, "w") as outf:
54 # outf.write(subset_defn_dims)
55 
56  # MUST snapshot if we're recursing further
57  if tier_num < n_tiers:
58  SAM.takeSnapshot(defname=output_def_name)
59 # print " " * (n_tiers - tier_num), "snapshot:", output_def_name
60 
61  group_defns.append(output_def_name)
62 
63  # if it's still too big, go around again...
64  if len(group_defns) > 1:
65  recurse_defns = Subdivide(group_defns)
66  return RecurseOverDefinitions(recurse_defns, tier_num+1, n_tiers, overlay_def, output_def_name_base)
67  else:
68  assert(tier_num == n_tiers), "Down to 1 group before all tiers completed?!"
69  return group_defns[0]
70 
71 
def RecurseOverDefinitions(input, tier_num, n_tiers, overlay_def, output_def_name_base, overwrite_defns=False)
assert(nhit_max >=nhit_nbins)
def overlay_prestage_def.Subdivide (   input)

Definition at line 21 of file overlay_prestage_def.py.

References PandAna.Demos.demo1.range.

Referenced by RecurseOverDefinitions().

21 def Subdivide(input):
22  return [ input[n*GROUP_SIZE:(n+1)*GROUP_SIZE] for n in range(len(input) / GROUP_SIZE + 1) ]
23 

Variable Documentation

overlay_prestage_def.action

Definition at line 90 of file overlay_prestage_def.py.

overlay_prestage_def.args

Definition at line 110 of file overlay_prestage_def.py.

overlay_prestage_def.check_output

Definition at line 159 of file overlay_prestage_def.py.

overlay_prestage_def.curr_perc

Definition at line 128 of file overlay_prestage_def.py.

overlay_prestage_def.default

Definition at line 91 of file overlay_prestage_def.py.

overlay_prestage_def.defname

Definition at line 121 of file overlay_prestage_def.py.

overlay_prestage_def.description

Definition at line 75 of file overlay_prestage_def.py.

overlay_prestage_def.dimensions

Definition at line 119 of file overlay_prestage_def.py.

overlay_prestage_def.experiment

Definition at line 72 of file overlay_prestage_def.py.

overlay_prestage_def.fhicl_files

Definition at line 115 of file overlay_prestage_def.py.

overlay_prestage_def.fhicl_runs

Definition at line 114 of file overlay_prestage_def.py.

overlay_prestage_def.file_list

Definition at line 119 of file overlay_prestage_def.py.

overlay_prestage_def.FILE_PATTERN

Definition at line 19 of file overlay_prestage_def.py.

overlay_prestage_def.files

Definition at line 161 of file overlay_prestage_def.py.

overlay_prestage_def.files_unmatched

Definition at line 171 of file overlay_prestage_def.py.

overlay_prestage_def.GROUP_SIZE

Definition at line 17 of file overlay_prestage_def.py.

overlay_prestage_def.help

Definition at line 78 of file overlay_prestage_def.py.

overlay_prestage_def.matches
overlay_prestage_def.md

Definition at line 139 of file overlay_prestage_def.py.

overlay_prestage_def.metavar

Definition at line 77 of file overlay_prestage_def.py.

overlay_prestage_def.n_files

Definition at line 123 of file overlay_prestage_def.py.

overlay_prestage_def.n_files_div10

Definition at line 124 of file overlay_prestage_def.py.

overlay_prestage_def.n_runs

Definition at line 169 of file overlay_prestage_def.py.

overlay_prestage_def.nargs

Definition at line 82 of file overlay_prestage_def.py.

overlay_prestage_def.num_tiers

Definition at line 149 of file overlay_prestage_def.py.

overlay_prestage_def.output_def_name

Definition at line 112 of file overlay_prestage_def.py.

overlay_prestage_def.overlay_runs

Definition at line 172 of file overlay_prestage_def.py.

overlay_prestage_def.parser

Definition at line 75 of file overlay_prestage_def.py.

overlay_prestage_def.SAM

Definition at line 72 of file overlay_prestage_def.py.

overlay_prestage_def.snapid

Definition at line 163 of file overlay_prestage_def.py.