> Predicting success and ingestion terminated normally. But score failed

data 14
score.py raise RuntimeError("no end.txt exist, ingestion failed")
I referred to the log, everything was going as its way：

2019-10-09 08:38:45,178 INFO ingestion.py: ===== Start ingestion program.
2019-10-09 08:38:45,178 INFO ingestion.py: ===== Initialize args.
2019-10-09 08:38:45,180 INFO ingestion.py: ===== Load metadata.
2019-10-09 08:38:45,181 INFO ingestion.py: Time budget: 1500
2019-10-09 08:38:45,181 INFO ingestion.py: ===== Set alive_thd
2019-10-09 08:38:45,181 INFO ingestion.py: ===== alive_thd started
2019-10-09 08:38:45,181 INFO ingestion.py: ===== Load user model
Info:
{'task': 'ssl', 'time_budget': 1500, 'schema': {'t_1': 'time', 'c_1': 'cat', 'c_2': 'cat', 'c_3': 'cat', 'c_4': 'cat', 'c_5': 'cat', 'c_6': 'cat', 'c_7': 'cat', 'c_8': 'cat', 'n_1': 'num', 'c_9': 'cat', 'n_2': 'num', 'n_3': 'num', 'n_4': 'num', 'n_5': 'num', 'n_6': 'num', 'n_7': 'num', 'c_10': 'cat', 'c_11': 'cat', 'c_12': 'cat', 'c_13': 'cat', 'c_14': 'cat', 'c_15': 'cat', 'c_16': 'cat', 'c_17': 'cat'}, 'pred_time_budget': 120}
Info:
{'task': 'ssl', 'time_budget': 1500, 'schema': {'t_1': 'time', 'c_1': 'cat', 'c_2': 'cat', 'c_3': 'cat', 'c_4': 'cat', 'c_5': 'cat', 'c_6': 'cat', 'c_7': 'cat', 'c_8': 'cat', 'n_1': 'num', 'c_9': 'cat', 'n_2': 'num', 'n_3': 'num', 'n_4': 'num', 'n_5': 'num', 'n_6': 'num', 'n_7': 'num', 'c_10': 'cat', 'c_11': 'cat', 'c_12': 'cat', 'c_13': 'cat', 'c_14': 'cat', 'c_15': 'cat', 'c_16': 'cat', 'c_17': 'cat'}, 'pred_time_budget': 120}
2019-10-09 08:38:46,097 INFO ingestion.py: ===== Check user model methods
2019-10-09 08:38:46,098 INFO ingestion.py: ===== Begin training user model

Start [train]:
----Start [clean_table]:
--------Start [clean_df]:
------------Start [fillna]:
------------End [fillna]. Time elapsed: 2.00 sec.
--------End [clean_df]. Time elapsed: 2.00 sec.
----End [clean_table]. Time elapsed: 2.00 sec.

----Start [feature_engineer]:
--------Start [transform_datetime]:
--------End [transform_datetime]. Time elapsed: 0.43 sec.

--------Start [categorical_encode]:
--------End [categorical_encode]. Time elapsed: 25.28 sec.

--------Start [numerical_encode]:
--------End [numerical_encode]. Time elapsed: 0.00 sec.
----End [feature_engineer]. Time elapsed: 25.71 sec.
----Remain time: 1472.2896468639374

End [train]. Time elapsed: 378.50 sec.

Start [save]:
End [save]. Time elapsed: 0.15 sec.
2019-10-09 08:45:29,598 INFO common.py: training success, time spent so far 378.65544605255127 sec
2019-10-09 08:45:29,598 INFO ingestion.py: Finished training the model. time spent 3.8e+02 sec
2019-10-09 08:45:29,599 INFO ingestion.py: ===== Begin preding by user model on test set
2019-10-09 08:45:29,599 INFO ingestion.py: ===== call subprocess of prediction
Info:
{'task': 'ssl', 'time_budget': 1500, 'schema': {'t_1': 'time', 'c_1': 'cat', 'c_2': 'cat', 'c_3': 'cat', 'c_4': 'cat', 'c_5': 'cat', 'c_6': 'cat', 'c_7': 'cat', 'c_8': 'cat', 'n_1': 'num', 'c_9': 'cat', 'n_2': 'num', 'n_3': 'num', 'n_4': 'num', 'n_5': 'num', 'n_6': 'num', 'n_7': 'num', 'c_10': 'cat', 'c_11': 'cat', 'c_12': 'cat', 'c_13': 'cat', 'c_14': 'cat', 'c_15': 'cat', 'c_16': 'cat', 'c_17': 'cat'}, 'pred_time_budget': 120}
Info:
{'task': 'ssl', 'time_budget': 1500, 'schema': {'t_1': 'time', 'c_1': 'cat', 'c_2': 'cat', 'c_3': 'cat', 'c_4': 'cat', 'c_5': 'cat', 'c_6': 'cat', 'c_7': 'cat', 'c_8': 'cat', 'n_1': 'num', 'c_9': 'cat', 'n_2': 'num', 'n_3': 'num', 'n_4': 'num', 'n_5': 'num', 'n_6': 'num', 'n_7': 'num', 'c_10': 'cat', 'c_11': 'cat', 'c_12': 'cat', 'c_13': 'cat', 'c_14': 'cat', 'c_15': 'cat', 'c_16': 'cat', 'c_17': 'cat'}, 'pred_time_budget': 120}

Start [load]:
End [load]. Time elapsed: 0.13 sec.

Start [predict]:
----Start [clean_table]:
--------Start [clean_df]:
------------Start [fillna]:
------------End [fillna]. Time elapsed: 2.01 sec.
--------End [clean_df]. Time elapsed: 2.01 sec.
----End [clean_table]. Time elapsed: 2.01 sec.

----Start [feature_engineer]:
--------Start [transform_datetime]:
--------End [transform_datetime]. Time elapsed: 0.46 sec.

--------Start [categorical_encode]:
--------End [categorical_encode]. Time elapsed: 22.48 sec.

--------Start [numerical_encode]:
--------End [numerical_encode]. Time elapsed: 0.00 sec.
----End [feature_engineer]. Time elapsed: 22.94 sec.
End [predict]. Time elapsed: 61.94 sec.
2019-10-09 08:46:58,393 INFO common.py: predicting success, time spent so far 87.34921383857727 sec
2019-10-09 08:47:05,047 INFO ingestion.py: Wrote the file end.txt marking the end of ingestion.
2019-10-09 08:47:05,048 INFO ingestion.py: [+] Done. Ingestion program successfully terminated.
2019-10-09 08:47:05,048 INFO ingestion.py: [+] Overall time spent 4.7e+02 sec
2019-10-09 08:47:05,111 INFO ingestion.py: [Ingestion terminated]

Posted by: coolshan008 @ Oct. 9, 2019, 5:31 p.m.

---

Dear participant,

Please reply that:
1. did you run locally using the starting kit or on the website when the problem occurred.
2. can you reproduce the problem stably？If yes:
3. please provide us with your submission by sending it to autowsl2019@4paradigm.com. (the full log is also useful if you test submission locally)

Thanks,
AutoWSL2019 Organizers

Posted by: liushouxiang @ Oct. 11, 2019, 9:24 a.m.