Public Member Functions | Protected Member Functions | Protected Attributes | Static Protected Attributes | List of all members
ana::WildcardSource Class Reference

File source based on a wildcard (glob) More...

#include "/cvmfs/nova.opensciencegrid.org/externals/cafanacore/v01.10/src/CAFAna/Core/WildcardSource.h"

Inheritance diagram for ana::WildcardSource:
ana::FileListSource ana::IFileSource

Public Member Functions

 WildcardSource (const std::string &wildcard, int stride=-1, int offset=-1, int limit=-1)
 
virtual ~WildcardSource ()
 
virtual TFile * GetNextFile () override
 Returns the next file in sequence, ready for reading. More...
 
int NFiles () const override
 May return -1 indicating the number of files is not known. More...
 
const std::vector< std::string > & GetFileNames () const
 

Protected Member Functions

std::vector< std::stringCheckedWildcard (const std::string &wildcard) const
 

Protected Attributes

std::vector< std::stringfFileNames
 The list of files. More...
 
std::vector< std::string >::iterator fIt
 Iterator into fFileNames. More...
 
std::vector< std::stringfRetry
 List of files that failed 1st attempt. More...
 
bool fInRetry
 Did we finish fFileNames and are now in fRetry? More...
 
TFile * fFile
 The most-recently-returned file. More...
 

Static Protected Attributes

static bool fgGotTickets = false
 Have we renewed our tickets? More...
 

Detailed Description

File source based on a wildcard (glob)

Definition at line 8 of file WildcardSource.h.

Constructor & Destructor Documentation

ana::WildcardSource::WildcardSource ( const std::string wildcard,
int  stride = -1,
int  offset = -1,
int  limit = -1 
)
Parameters
wildcardWildcard or glob. Anything glob() accepts is OK. May be a single literal filename.

Definition at line 27 of file WildcardSource.cxx.

30  {
31  }
FileListSource(const std::vector< std::string > &files, int stride=-1, int offset=-1, int limit=-1)
Default stride, offset, and limit mean obey cmd-line options.
std::vector< std::string > CheckedWildcard(const std::string &wildcard) const
std::vector< std::string > wildcard(const std::string &wildcardString)
Definition: convert.C:9
ana::WildcardSource::~WildcardSource ( )
virtual

Definition at line 34 of file WildcardSource.cxx.

References CheckedWildcard().

35  {
36  }

Member Function Documentation

std::vector< std::string > ana::WildcardSource::CheckedWildcard ( const std::string wildcard) const
protected

Definition at line 40 of file WildcardSource.cxx.

References om::cout, allTimeWatchdog::endl, runNovaSAM::ret, and ana::Wildcard().

Referenced by ~WildcardSource().

41  {
42  std::vector<std::string> ret = Wildcard(wildcard);
43 
44  struct stat ss;
45  // If we found nothing, it may be because pnfs isn't mounted.
46  if(ret.empty() &&
47  wildcard.find("/pnfs/") == 0 &&
48  stat("/pnfs/", &ss) != 0){
49 
50  std::cout << "No files matching " << wildcard << " but that's probably because /pnfs is not mounted on the current grid node. If you have to use your own files, either try running it interactivly or try sam_add_dataset to create one." << std::endl;
51  }
52 
53  return ret;
54  }
Float_t ss
Definition: plot.C:24
std::vector< std::string > Wildcard(const std::string &wildcardString)
Find files matching a UNIX glob, plus expand environment variables.
Definition: UtilsExt.cxx:268
std::vector< std::string > wildcard(const std::string &wildcardString)
Definition: convert.C:9
OStream cout
Definition: OStream.cxx:6
const std::vector<std::string>& ana::FileListSource::GetFileNames ( ) const
inlineinherited

Definition at line 22 of file FileListSource.h.

References ana::FileListSource::fFileNames.

22 { return fFileNames; }
std::vector< std::string > fFileNames
The list of files.
TFile * ana::FileListSource::GetNextFile ( )
overridevirtualinherited

Returns the next file in sequence, ready for reading.

A null return means that the end of the sequence has been reached. DO NOT close or delete the file that is returned.

Implements ana::IFileSource.

Definition at line 78 of file FileListSource.cxx.

References ana::assert(), om::cout, allTimeWatchdog::endl, ana::FileListSource::fFile, ana::FileListSource::fFileNames, ana::FileListSource::fInRetry, ana::FileListSource::fIt, ana::FileListSource::fRetry, MECModelEnuComparisons::i, PandAna.Demos.demo0::loc, ana::pnfs2xrootd(), and string.

Referenced by make_dst_cosrejbdttrain(), trimmubarid(), and trimvar().

79  {
80  // Tidy up the last file we gave, which the caller no longer needs
81  delete fFile;
82  fFile = 0;
83 
84  // Did we run out of files?
85  if(fInRetry && fIt == fRetry.end()) return 0;
86  if(fIt == fFileNames.end()){
87  if(fRetry.empty()) return 0;
88  fIt = fRetry.begin();
89  fInRetry = true;
90  }
91 
92  // If the file is on pnfs rewrite it to an xrootd address
93  std::string loc = *fIt;
94  loc = pnfs2xrootd(loc); // no-op for non /pnfs locations
95 
96  if(fInRetry) std::cout << "Retrying " << loc << " which was previously deferred..." << std::endl;
97 
98  fFile = TFile::Open(loc.c_str()); // This pattern allows xrootd
99 
100  if(!fFile){
101  if(!fInRetry){
102  std::cout << "Unable to open " << loc << std::endl;
103  std::cout << "Will skip this file and try again later." << std::endl;
104  fRetry.push_back(*fIt);
105  ++fIt;
106  return GetNextFile();
107  }
108  else{
109  const int Nretry = 3;
110  std::cout << "Failed to open. Will retry " << Nretry << " more times" << std::endl;
111  for(int i = 0; i < Nretry; ++i){
112  std::cout << "Attempt " << i << std::endl;
113  fFile = TFile::Open(loc.c_str());
114  if(fFile){
115  std::cout << "Success!" << std::endl;
116  break;
117  }
118  } // end for i
119  if(!fFile){
120  std::cout << "Still unable to read " << loc << std::endl;
121  std::cout << "Aborting" << std::endl;
122  abort();
123  }
124  }
125  }
126 
127  // The above should have guaranteed this
128  assert(fFile);
129 
130  ++fIt; // Move on to the next file, for the subsequent call
131 
132  return fFile;
133  }
std::vector< std::string >::iterator fIt
Iterator into fFileNames.
std::string pnfs2xrootd(std::string loc, bool unauth)
Definition: UtilsExt.cxx:237
std::vector< std::string > fFileNames
The list of files.
bool fInRetry
Did we finish fFileNames and are now in fRetry?
OStream cout
Definition: OStream.cxx:6
std::vector< std::string > fRetry
List of files that failed 1st attempt.
virtual TFile * GetNextFile() override
Returns the next file in sequence, ready for reading.
assert(nhit_max >=nhit_nbins)
TFile * fFile
The most-recently-returned file.
enum BeamMode string
int ana::FileListSource::NFiles ( ) const
inlineoverridevirtualinherited

May return -1 indicating the number of files is not known.

Reimplemented from ana::IFileSource.

Definition at line 20 of file FileListSource.h.

References ana::FileListSource::fFileNames.

Referenced by trimmubarid(), trimvar(), and ana::SpectrumLoaderBase::WildcardOrSAMQuery().

20 {return fFileNames.size();}
std::vector< std::string > fFileNames
The list of files.

Member Data Documentation

TFile* ana::FileListSource::fFile
protectedinherited

The most-recently-returned file.

Definition at line 28 of file FileListSource.h.

Referenced by ana::FileListSource::GetNextFile(), and ana::FileListSource::~FileListSource().

std::vector<std::string> ana::FileListSource::fFileNames
protectedinherited
bool ana::FileListSource::fgGotTickets = false
staticprotectedinherited

Have we renewed our tickets?

Definition at line 29 of file FileListSource.h.

Referenced by ana::FileListSource::FileListSource().

bool ana::FileListSource::fInRetry
protectedinherited

Did we finish fFileNames and are now in fRetry?

Definition at line 27 of file FileListSource.h.

Referenced by ana::FileListSource::GetNextFile().

std::vector<std::string>::iterator ana::FileListSource::fIt
protectedinherited

Iterator into fFileNames.

Definition at line 25 of file FileListSource.h.

Referenced by ana::FileListSource::FileListSource(), and ana::FileListSource::GetNextFile().

std::vector<std::string> ana::FileListSource::fRetry
protectedinherited

List of files that failed 1st attempt.

Definition at line 26 of file FileListSource.h.

Referenced by ana::FileListSource::GetNextFile().


The documentation for this class was generated from the following files: