Wait, wouldn't this break if the program uses just SQL and no Python kernel? If they can't move it to PySpark, option B wouldn't apply. Are we sure Python is allowed here, or is there a native SQL scheduling trick in Databricks I missed?
B for sure. Wrapping with PySpark lets you use Python's datetime to check if it's Sunday, so you can conditionally run the last query. All the other options either restrict access or change what runs each day, which isn't what's needed here. Pretty confident on this but open to other ideas if I missed something.
Not D, B. Wrapping with PySpark lets you control exactly when the last query runs, unlike D which just restricts access (not really what’s needed here). If the requirement was to only run the program on Sundays, C would make sense. Was the daily schedule a must?
Had something like this in a mock. B lets you use Python control flow to hit that schedule requirement, just checking the day before running the last query. None of the others give the same flexibility, I think.