--- running make second ana page
--- Start date:
Wed Jul 24 02:50:01 CDT 2019
--- setup
--- Setup NOvA production validation
User: [01;34mnovapro[0m
Use with SRT: [01;34mfalse[0m
Base directory: [01;34m/nova/app/users/novapro/prod_valid/python[0m
Grid output: [01;34m/nova/ana/users/novapro/prod_valid/condor_output[0m
Web output: [01;34m/nusoft/app/web/htdoc/nova/production[0m
--- run
Fetching data for prod_artdaq_S15-03-11_fd_cosmic_period1_quarter
Fetching data for prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_period1_quarter
Fetching data for prod_pcliststop_R16-01-27-prod2calib.a_fd_cosmic_period1_quarter
Fetching data for prod_timecal_R16-01-27-prod2calib.a_fd_cosmic_period1_quarter
HTTP error: 502 Bad Gateway: retrying in 1 s
User dimensions query limit for novapro of 10 exceeded: retrying in 2 s
User dimensions query limit for novapro of 10 exceeded: retrying in 4 s
User dimensions query limit for novapro of 10 exceeded: retrying in 8 s
User dimensions query limit for novapro of 10 exceeded: retrying in 16 s
User dimensions query limit for novapro of 10 exceeded: retrying in 32 s
User dimensions query limit for novapro of 10 exceeded: retrying in 60 s
User dimensions query limit for novapro of 10 exceeded: retrying in 60 s
User dimensions query limit for novapro of 10 exceeded: retrying in 60 s
User dimensions query limit for novapro of 10 exceeded: retrying in 60 s
User dimensions query limit for novapro of 10 exceeded: retrying in 60 s
User dimensions query limit for novapro of 10 exceeded: retrying in 60 s
User dimensions query limit for novapro of 10 exceeded: retrying in 60 s
User dimensions query limit for novapro of 10 exceeded: retrying in 60 s
Fetching data for prod_artdaq_S15-03-11_fd_cosmic_period2_quarter
(psycopg2.OperationalError) server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
[SQL: 'SELECT count(summary.file_id) AS count_1, sum(summary.file_size_in_bytes) AS sum_1, sum(summary.event_count) AS sum_2 \nFROM (SELECT DISTINCT ON (with_rownum.file_id) with_rownum.file_id AS file_id, with_rownum.file_size_in_bytes AS file_size_in_bytes, with_rownum.event_count AS event_count \nFROM (SELECT data_files.file_id AS file_id, data_files.file_size_in_bytes AS file_size_in_bytes, data_files.event_count AS event_count, dense_rank() OVER (ORDER BY data_files.file_id) AS rownum \nFROM data_files JOIN num_data_files_param_values AS online_runnumber ON data_files.file_id = online_runnumber.file_id JOIN data_files_param_values AS dfpv_nova_detectorid ON data_files.file_id = dfpv_nova_detectorid.file_id JOIN file_types ON file_types.file_type_id = data_files.file_type_id JOIN num_data_files_param_values AS online_totalevents ON data_files.file_id = online_totalevents.file_id JOIN data_files_param_values AS dfpv_nova_standard ON data_files.file_id = dfpv_nova_standard.file_id JOIN data_files_param_values AS dfpv_online_partition ON data_files.file_id = dfpv_online_partition.file_id JOIN data_files_param_values AS dfpv_online_stream ON data_files.file_id = dfpv_online_stream.file_id JOIN data_tiers ON data_tiers.data_tier_id = data_files.data_tier_id JOIN data_files_param_values AS dfpv_daq2rawdigit_base_release ON data_files.file_id = dfpv_daq2rawdigit_base_release.file_id \nWHERE online_runnumber.param_type_id = 17 AND online_runnumber.param_value >= %(param_value_1)s AND online_runnumber.param_type_id = 17 AND online_runnumber.param_value <= %(param_value_2)s AND dfpv_nova_detectorid.param_type_id = 86 AND dfpv_nova_detectorid.param_value_id = %(param_value_id_1)s AND lower(file_types.file_type_desc) = %(param_1)s AND online_totalevents.param_type_id = 25 AND online_totalevents.param_value > %(param_value_3)s AND dfpv_nova_standard.param_type_id = 84 AND dfpv_nova_standard.param_value_id = %(param_value_id_2)s AND dfpv_online_partition.param_type_id = 13 AND dfpv_online_partition.param_value_id = %(param_value_id_3)s AND dfpv_online_stream.param_type_id = 21 AND dfpv_online_stream.param_value_id = %(param_value_id_4)s AND data_tiers.data_tier = %(data_tier_1)s AND dfpv_daq2rawdigit_base_release.param_type_id = 108 AND dfpv_daq2rawdigit_base_release.param_value_id = %(param_value_id_5)s AND data_files.retired_date IS NULL AND data_files.file_content_status_id = 2 AND (EXISTS (SELECT 1 \nFROM data_file_locations \nWHERE data_file_locations.file_id = data_files.file_id))) AS with_rownum \nWHERE (with_rownum.rownum - 1) %% %(param_2)s = %(param_3)s ORDER BY with_rownum.file_id) AS summary'] [parameters: {'param_value_id_2': 363059L, 'param_value_id_3': 142157L, 'param_value_id_1': 459309L, 'param_value_id_4': 13422L, 'param_value_id_5': 2042266L, 'data_tier_1': 'artdaq', 'param_1': 'importeddetector', 'param_3': 0, 'param_2': 4, 'param_value_1': 17967, 'param_value_3': 0, 'param_value_2': 19746}] (Background on this error at: //sqlalche.me/e/e3q8): retrying in 1 s
(psycopg2.OperationalError) FATAL: the database system is in recovery mode
(Background on this error at: //sqlalche.me/e/e3q8): retrying in 2 s
(psycopg2.OperationalError) FATAL: the database system is in recovery mode
(Background on this error at: //sqlalche.me/e/e3q8): retrying in 4 s
(psycopg2.OperationalError) FATAL: the database system is in recovery mode
(Background on this error at: //sqlalche.me/e/e3q8): retrying in 8 s
HTTP error: 502 Bad Gateway: retrying in 16 s
User dimensions query limit for novapro of 10 exceeded: retrying in 32 s
Fetching data for prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_period2_quarter
Fetching data for prod_pcliststop_R16-01-27-prod2calib.a_fd_cosmic_period2_quarter
Fetching data for prod_timecal_R16-01-27-prod2calib.a_fd_cosmic_period2_quarter
Fetching data for prod_artdaq_S15-03-11_fd_cosmic_epoch3b
Fetching data for prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_epoch3b
Fetching data for prod_pcliststop_R16-01-27-prod2calib.a_fd_cosmic_epoch3b
Fetching data for prod_timecal_R16-01-27-prod2calib.a_fd_cosmic_epoch3b
Fetching data for prod_artdaq_S15-03-11_fd_cosmic_epoch3c
Fetching data for prod_pclist_R16-01-27-prod2calib.a_fd_cosmic_epoch3c
Fetching data for prod_pcliststop_R16-01-27-prod2calib.a_fd_cosmic_epoch3c
Fetching data for prod_timecal_R16-01-27-prod2calib.a_fd_cosmic_epoch3c
Fetching data for prod_fcl_v01.04_fd_cry_all_nova_v08_ideal_batch1_v1_gain100
Fetching data for prod_daq_R16-01-27-prod2calib.b_fd_cry_all_nova_v08_ideal_batch1_v1_gain100
Fetching data for prod_pclist_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain100
Fetching data for prod_pcliststop_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain100
Fetching data for prod_timecal_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain100
Fetching data for prod_fcl_v01.04_fd_cry_all_nova_v08_ideal_batch1_v1_gain140
Fetching data for prod_daq_R16-01-27-prod2calib.b_fd_cry_all_nova_v08_ideal_batch1_v1_gain140
Fetching data for prod_pclist_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain140
Fetching data for prod_pcliststop_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain140
Fetching data for prod_timecal_R16-01-27-prod2calib.e_fd_cry_all_nova_v08_ideal_batch1_v1_gain140
Fetching data for prod_artdaq_S16-02-02_nd_cosmic_period1
Fetching data for prod_pclist_R16-01-27-prod2calib.c_nd_cosmic_period1
Fetching data for prod_pcliststop_R16-01-27-prod2calib.c_nd_cosmic_period1
Fetching data for prod_timecal_R16-01-27-prod2calib.c_nd_cosmic_period1
Fetching data for prod_artdaq_S16-02-02_nd_cosmic_period2
Fetching data for prod_pclist_R16-01-27-prod2calib.c_nd_cosmic_period2
Fetching data for prod_pcliststop_R16-01-27-prod2calib.c_nd_cosmic_period2
Fetching data for prod_timecal_R16-01-27-prod2calib.c_nd_cosmic_period2
Fetching data for prod_artdaq_S16-02-02_nd_cosmic_epoch3b
Fetching data for prod_pclist_R16-01-27-prod2calib.c_nd_cosmic_epoch3b
Fetching data for prod_pcliststop_R16-01-27-prod2calib.c_nd_cosmic_epoch3b
Fetching data for prod_timecal_R16-01-27-prod2calib.c_nd_cosmic_epoch3b
Fetching data for prod_artdaq_S16-02-02_nd_cosmic_epoch3c
Fetching data for prod_pclist_R16-01-27-prod2calib.c_nd_cosmic_epoch3c
Fetching data for prod_pcliststop_R16-01-27-prod2calib.c_nd_cosmic_epoch3c
Fetching data for prod_timecal_R16-01-27-prod2calib.c_nd_cosmic_epoch3c
Fetching data for prod_artdaq_S16-02-02_nd_ddactivity_period2
Fetching data for prod_pclist_R16-01-27-prod2calib.e_nd_ddactivity_period2_v2
HTTP error: 502 Bad Gateway: retrying in 1 s
Fetching data for prod_pcliststop_R16-01-27-prod2calib.e_nd_ddactivity_period2_v2
Fetching data for prod_timecal_R16-01-27-prod2calib.e_nd_ddactivity_period2_v2
Fetching data for prod_artdaq_S16-02-02_nd_ddactivity_epoch3b
Fetching data for prod_pclist_R16-01-27-prod2calib.e_nd_ddactivity_epoch3b_v3
Fetching data for prod_pcliststop_R16-01-27-prod2calib.e_nd_ddactivity_epoch3b_v3
Fetching data for prod_timecal_R16-01-27-prod2calib.e_nd_ddactivity_epoch3b_v3
Fetching data for prod_artdaq_S16-02-02_nd_ddactivity_epoch3c
Fetching data for prod_pclist_R16-01-27-prod2calib.e_nd_ddactivity_epoch3c_v2
Fetching data for prod_pcliststop_R16-01-27-prod2calib.e_nd_ddactivity_epoch3c_v2
Fetching data for prod_timecal_R16-01-27-prod2calib.e_nd_ddactivity_epoch3c_v2
Fetching data for prod_fcl_v01.04_nd_cry_all_nova_v08_ideal_batch1_v1
Fetching data for prod_artdaq_R16-01-27-prod2calib.c_none
Fetching data for prod_pclist_R16-01-27-prod2calib.e_none
Fetching data for prod_pcliststop_R16-01-27-prod2calib.e_none
Fetching data for prod_timecal_R16-01-27-prod2calib.e_none
--- done
--- End date:
Wed Jul 24 07:18:57 CDT 2019
---
