DAG: SierraGorda_ParametersStorage Functions that calculate parameters like fragmentation in SierraGorda

schedule: 45,45 12,0 * * *


Task Instance: fragmentation_AddBenchToWenco


Task Instance Details

Dependencies Blocking Task From Getting Scheduled
Dependency Reason
Trigger Rule Task's trigger rule 'all_success' requires all upstream tasks to have succeeded, but found 1 non-success(es). upstream_tasks_state={'total': 1, 'successes': 0, 'skipped': 0, 'failed': 1, 'upstream_failed': 0, 'done': 1}, upstream_task_ids={'fragmentation_joinWencoFiles'}
Dagrun Running Task instance's dagrun was not in the 'running' state but in the state 'failed'.
Task Instance State Task is in the 'upstream_failed' state which is not a valid state for execution. The task must be cleared in order to be run.
Attribute: python_callable
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
def agregar_banco_a_wenco(origin_wenco_file_container, origin_opit_container, origin_opit_blob, origin_wenco_file_blob, final_container, final_blob): 
    """
    Función que permite agregar banco a datos en wenco
    
    Parámetros
    ----------
    origin_container: str
        Nombre del contenedor donde se encuentra el archivo a limpiar.
    origin_hole_position_blob: str
        Nombre del archivo que contiene la posición de los pozos.
    origin_hole_information_blob: str
        Nombre del archivo que contiene la información de los pozos.
    origin_hole_profile_blob: str
        Nombre del archivo que contiene el perfil de los pozos.
    final_container: str
        Nombre del contenedor donde se guardará el archivo limpio.
    final_blob: str
        Nombre del archivo que se guardará en el contenedor final.

    Returns
    -------
    None
    """
    # Archivo Wenco Shovel
    conn_origin_container_wenco = BlobClient.from_connection_string(conn_str=BLOB_CONNECT_STRING, container_name=origin_wenco_file_container, blob_name=origin_wenco_file_blob)
    download_stream_wenco = conn_origin_container_wenco.download_blob()
    df_wenco = pd.read_json(download_stream_wenco, orient='table')  

    # Archivo opit 
    conn_origin_containerOpit = BlobClient.from_connection_string(conn_str=BLOB_CONNECT_STRING, container_name=origin_opit_container, blob_name=origin_opit_blob)
    download_streamOpit = conn_origin_containerOpit.download_blob()

    stream = BytesIO()
    download_streamOpit.readinto(stream)
    df_opit = pd.read_parquet(stream)

    df_opit = df_opit.loc[(df_opit['holes_tipo']=='R')]
    df_opit = df_opit.loc[~df_opit['Bank'].str.contains('[a-zA-Z]')]
    df_opit = df_opit.loc[df_opit['Bank']!='']
    df_opit['Bank'] = df_opit['Bank'].astype(int)

    # Preparar cruce de datos
    df_wenco['BUCKET_ELEVATION'] = df_wenco['BUCKET_ELEVATION'].astype(float)
    lista_bancos = df_opit['Bank'].unique()

    df_wenco_banco = pd.DataFrame(columns=df_wenco.columns)
    for banco in lista_bancos: 
        try: 
            banco = int(banco)
            print(f'Procesando banco: {banco}')
            df_wenco_banco_aux = df_wenco[(df_wenco['BUCKET_ELEVATION'] >= banco - 3) & (df_wenco['BUCKET_ELEVATION'] <= banco + 8)]
            df_wenco_banco_aux['Bank'] = banco

            df_wenco_banco = pd.concat([df_wenco_banco, df_wenco_banco_aux], ignore_index=True)
        except Exception as e:
            print(f'No se pudo procesar banco: {banco}. Error: {e}')
            continue

    # Configuracion para subir al blob
    blob = BlobClient.from_connection_string(conn_str=BLOB_CONNECT_STRING, container_name=final_container, blob_name=final_blob)
    ## Subir la data 
    json_data = df_wenco_banco.to_json(index=False, orient='table')
    blob.upload_blob(json_data, overwrite=True)
Task Instance Attributes
Attribute Value
dag_id SierraGorda_ParametersStorage
duration None
end_date 2026-06-13 00:46:17.059229+00:00
execution_date 2026-06-12T12:45:00+00:00
executor_config {}
generate_command <function TaskInstance.generate_command at 0x72aa98ce3040>
hostname
is_premature False
job_id None
key ('SierraGorda_ParametersStorage', 'fragmentation_AddBenchToWenco', <Pendulum [2026-06-12T12:45:00+00:00]>, 1)
log <Logger airflow.task (INFO)>
log_filepath /usr/local/airflow/logs/SierraGorda_ParametersStorage/fragmentation_AddBenchToWenco/2026-06-12T12:45:00+00:00.log
log_url http://localhost:8080/admin/airflow/log?execution_date=2026-06-12T12%3A45%3A00%2B00%3A00&task_id=fragmentation_AddBenchToWenco&dag_id=SierraGorda_ParametersStorage
logger <Logger airflow.task (INFO)>
mark_success_url http://localhost:8080/success?task_id=fragmentation_AddBenchToWenco&dag_id=SierraGorda_ParametersStorage&execution_date=2026-06-12T12%3A45%3A00%2B00%3A00&upstream=false&downstream=false
max_tries 0
metadata MetaData(bind=None)
next_try_number 1
operator None
pid None
pool default_pool
prev_attempted_tries 0
previous_execution_date_success None
previous_start_date_success None
previous_ti <TaskInstance: SierraGorda_ParametersStorage.fragmentation_AddBenchToWenco 2026-06-12 00:45:00+00:00 [upstream_failed]>
previous_ti_success None
priority_weight 4
queue default
queued_dttm None
raw False
run_as_user None
start_date 2026-06-13 00:46:17.059208+00:00
state upstream_failed
task <Task(PythonOperator): fragmentation_AddBenchToWenco>
task_id fragmentation_AddBenchToWenco
test_mode False
try_number 1
unixname airflow
Task Attributes
Attribute Value
dag <DAG: SierraGorda_ParametersStorage>
dag_id SierraGorda_ParametersStorage
depends_on_past False
deps {<TIDep(Not In Retry Period)>, <TIDep(Trigger Rule)>, <TIDep(Previous Dagrun State)>}
do_xcom_push True
downstream_list [<Task(PythonOperator): fragmentation_AddExcavationRate>]
downstream_task_ids {'fragmentation_AddExcavationRate'}
email None
email_on_failure True
email_on_retry True
end_date None
execution_timeout None
executor_config {}
extra_links []
global_operator_extra_link_dict {}
inlets []
lineage_data None
log <Logger airflow.task.operators (INFO)>
logger <Logger airflow.task.operators (INFO)>
max_retry_delay None
on_failure_callback None
on_retry_callback None
on_success_callback None
op_args []
op_kwargs {'origin_wenco_file_container': 'processed', 'origin_opit_container': 'raw', 'origin_wenco_file_blob': 'SierraGorda/Fragmentacion/2026-06-13/WencoShovelHaul.json', 'origin_opit_blob': 'SierraGorda/Historico/opit.parquet', 'final_container': 'processed', 'final_blob': 'SierraGorda/Fragmentacion/2026-06-13/WencoShovelHaul_banco.json'}
operator_extra_link_dict {}
operator_extra_links ()
outlets []
owner pedro
params {}
pool default_pool
priority_weight 1
priority_weight_total 4
provide_context False
queue default
resources None
retries 0
retry_delay 0:05:00
retry_exponential_backoff False
run_as_user None
schedule_interval 45,45 12,0 * * *
shallow_copy_attrs ('python_callable', 'op_kwargs')
sla None
start_date 2023-04-22T00:00:00+00:00
subdag None
task_concurrency None
task_id fragmentation_AddBenchToWenco
task_type PythonOperator
template_ext []
template_fields ('templates_dict', 'op_args', 'op_kwargs')
templates_dict None
trigger_rule all_success
ui_color #ffefeb
ui_fgcolor #000
upstream_list [<Task(PythonOperator): fragmentation_joinWencoFiles>]
upstream_task_ids {'fragmentation_joinWencoFiles'}
wait_for_downstream False
weight_rule downstream