Tag: <span>Workflow Management</span>

ETL process usually involves a few discrete tasks that relies on each other. One way to achieve this is to use crontab (or Kubernetes CronJob) where each task is scheduled to run at a specific time. However, ensuring inter-dependencies among the tasks, where one task should only start when the previous has finished is not…

Data Engineering