prefect dag scheduling — cloud

October 20, 2020

trying out prefect cloud features to determine deployment strategy for productionalizing workflows

• logged into prefect cloud
• created a new project in the Team section:

• then Projects :

• Navigated to tutorials:

• Went through the steps of selecting the project, deploying a flow, setting local environment viapip install prefect -U
• Logging into Prefect Cloud via CLI — need to create access token

when i try to authenticate with token in CLI via:

prefect auth login -t <token>

i get error:

Error: Auth commands with server are not currently supported.

looks like i need to run update to use prefect backend cloud (unlike when i ran prefect backend server previously to run my flows locally)

you should see a message that says

Backend switched to cloud

after running prefect auth login -t <token> you should see:

Login successful!

• next we run docker agent locally, we need to create an agent or Runner token, then you can run

prefect agent start docker -t <agent-token>

you should see something like this:

for the tutorial, there is a docker image with the tutorial flow. when starting the flow run for it you should see:

next i’m trying out universal deploy, i think this is meant to show how easily you can deploy a flow to cloud after creating a flow locally.

if you define your tasks and flow as previously, you can add at the bottom:

flow.run()
flow.register(project_name="<project-name>")
flow.run_agent(token='<agent-token>')

it will deploy the flow to the cloud

next i’m trying out the graphql api to query our flows and potentially edit our flows and re-register. once you authenticate with token via CLI, you can use the prefect python Client to make graphql queries. you can also run queries directly in the interactive API in the browser logged in.

although it appears you can edit workflows via the graphql api, i’m not confident that we would be able to avoid making incremental changes to an existing workflow due to the dag versioning feature of flows in prefect.