overlay_prestage_def Namespace Reference

Functions
def	Subdivide (input)
def	RecurseOverDefinitions (input, tier_num, n_tiers, overlay_def, output_def_name_base, overwrite_defns=False)

Variables
	GROUP_SIZE
	FILE_PATTERN
	SAM
	experiment
	parser
	description
	metavar
	help
	nargs
	action
	default
	args
	output_def_name
	fhicl_runs
	fhicl_files
	file_list
	dimensions
	defname
	n_files
	n_files_div10
	curr_perc
	matches
	md
	num_tiers
	check_output
	files
	snapid
	n_runs
	files_unmatched
	overlay_runs

Detailed Description

  overlay_prestage_def.py:
  
  Create (and snapshot) a SAM definition that contains only those files
  from an overlay definition corresponding to the files actually used
  by the overlay FCLs in the FCL definition. 

Function Documentation

def overlay_prestage_def.RecurseOverDefinitions	(		input,
			tier_num,
			n_tiers,
			overlay_def,
			output_def_name_base,
			overwrite_defns = False
	)

Definition at line 24 of file overlay_prestage_def.py.

References ana.assert(), and Subdivide().

def RecurseOverDefinitions(input, tier_num, n_tiers, overlay_def, output_def_name_base, overwrite_defns=False):
    if len(input) == 0:
        return None
    
    group_defns = []
    for group_idx, grouping in enumerate(input):
        
        # fhicl_runs (they're tuples of two values)
        if all(len(g) == 2 and all(isinstance(i, int) for i in g) for g in grouping):
            subset_defn_dims = "defname: %s and (%s)" % (overlay_def, " or ".join(("run_number %d.%d" % (run,sub) for run,sub in grouping)))
        else:
            subset_defn_dims = " or ".join("dataset_def_name_newest_snapshot %s" % g for g in grouping)

        if tier_num < n_tiers:
            suffix = "_" + ("Sub" * (n_tiers - tier_num)) + ("set%d" % group_idx)
        else:
            suffix = ""
        
        output_def_name = output_def_name_base + suffix
        
        if overwrite_defns:
            try:
                SAM.deleteDefinition(defname=output_def_name)
            except samweb_client._exceptions.DefinitionNotFound:
                pass

        SAM.createDefinition(defname=output_def_name, dims=subset_defn_dims)
                
#         print "  " * (n_tiers - tier_num), "new defn:", output_def_name
#         with open(output_def_name, "w") as outf:
#             outf.write(subset_defn_dims)
        
        # MUST snapshot if we're recursing further
        if tier_num < n_tiers:
            SAM.takeSnapshot(defname=output_def_name)
#             print "  " * (n_tiers - tier_num), "snapshot:", output_def_name

        group_defns.append(output_def_name)
    
    # if it's still too big, go around again...
    if len(group_defns) > 1:
        recurse_defns = Subdivide(group_defns)
        return RecurseOverDefinitions(recurse_defns, tier_num+1, n_tiers, overlay_def, output_def_name_base)
    else:
        assert(tier_num == n_tiers), "Down to 1 group before all tiers completed?!"
        return group_defns[0]
   

def overlay_prestage_def.Subdivide

(

input

)

Definition at line 21 of file overlay_prestage_def.py.

References PandAna.Demos.demo1.range.

Referenced by RecurseOverDefinitions().

def Subdivide(input):
    return [ input[n*GROUP_SIZE:(n+1)*GROUP_SIZE] for n in range(len(input) / GROUP_SIZE + 1) ]

Variable Documentation

overlay_prestage_def.action

Definition at line 90 of file overlay_prestage_def.py.

overlay_prestage_def.args

Definition at line 110 of file overlay_prestage_def.py.

overlay_prestage_def.check_output

Definition at line 159 of file overlay_prestage_def.py.

overlay_prestage_def.curr_perc

Definition at line 128 of file overlay_prestage_def.py.

overlay_prestage_def.default

Definition at line 91 of file overlay_prestage_def.py.

overlay_prestage_def.defname

Definition at line 121 of file overlay_prestage_def.py.

overlay_prestage_def.description

Definition at line 75 of file overlay_prestage_def.py.

overlay_prestage_def.dimensions

Definition at line 119 of file overlay_prestage_def.py.

overlay_prestage_def.experiment

Definition at line 72 of file overlay_prestage_def.py.

overlay_prestage_def.fhicl_files

Definition at line 115 of file overlay_prestage_def.py.

overlay_prestage_def.fhicl_runs

Definition at line 114 of file overlay_prestage_def.py.

overlay_prestage_def.file_list

Definition at line 119 of file overlay_prestage_def.py.

overlay_prestage_def.FILE_PATTERN

Definition at line 19 of file overlay_prestage_def.py.

overlay_prestage_def.files

Definition at line 161 of file overlay_prestage_def.py.

overlay_prestage_def.files_unmatched

Definition at line 171 of file overlay_prestage_def.py.

overlay_prestage_def.GROUP_SIZE

Definition at line 17 of file overlay_prestage_def.py.

overlay_prestage_def.help

Definition at line 78 of file overlay_prestage_def.py.

overlay_prestage_def.matches

Definition at line 135 of file overlay_prestage_def.py.

Referenced by novarwgt.ConvertSRTruth(), genie::TuneId.Decode(), mono::MonopoleTrack.filter(), lem::FindMatchesAlg.GetLibrary(), stan::lang.is_user_defined(), load_libs_from_command(), LZ4HC_compress_optimal(), art::detail.print_available_plugins(), lem::MergeMatches.produce(), lem::MakeMatches.produce(), lem::FindLEMMatches.produce(), lem::MakePID.produce(), lem::LEM.produce(), set_eventn_status_hit(), and cheat::BackTracker.TracksToParticles().

overlay_prestage_def.md

Definition at line 139 of file overlay_prestage_def.py.

overlay_prestage_def.metavar

Definition at line 77 of file overlay_prestage_def.py.

overlay_prestage_def.n_files

Definition at line 123 of file overlay_prestage_def.py.

overlay_prestage_def.n_files_div10

Definition at line 124 of file overlay_prestage_def.py.

overlay_prestage_def.n_runs

Definition at line 169 of file overlay_prestage_def.py.

overlay_prestage_def.nargs

Definition at line 82 of file overlay_prestage_def.py.

overlay_prestage_def.num_tiers

Definition at line 149 of file overlay_prestage_def.py.

overlay_prestage_def.output_def_name

Definition at line 112 of file overlay_prestage_def.py.

overlay_prestage_def.overlay_runs

Definition at line 172 of file overlay_prestage_def.py.

overlay_prestage_def.parser

Definition at line 75 of file overlay_prestage_def.py.

overlay_prestage_def.SAM

Definition at line 72 of file overlay_prestage_def.py.

overlay_prestage_def.snapid

Definition at line 163 of file overlay_prestage_def.py.