{"id":9304,"date":"2021-05-31T05:31:21","date_gmt":"2021-05-31T05:31:21","guid":{"rendered":"https:\/\/prwatech.in\/blog\/?p=9304"},"modified":"2024-03-19T09:35:01","modified_gmt":"2024-03-19T09:35:01","slug":"working-with-composer-using-airflow","status":"publish","type":"post","link":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/","title":{"rendered":"Working with composer using airflow"},"content":{"rendered":"\r\n<h2><span data-sheets-root=\"1\" data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;&lt;yoastmark class=\">Cloud Composer and Airflow Integration<\/span><\/h2>\r\n<p><strong>Prerequisites<\/strong><\/p>\r\n\r\n\r\n\r\n<p><a href=\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/gcp-google-cloud-platform\/\" target=\"_blank\" rel=\"noreferrer noopener\" data-type=\"URL\" data-id=\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/gcp-google-cloud-platform\/\">GCP<\/a> account<\/p>\r\n\r\n\r\n\r\n<p>Open Console<\/p>\r\n\r\n\r\n\r\n<p>Open Menu &gt; Cloud Storage &gt; Browser<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"392\" height=\"269\" class=\"wp-image-9309\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png 392w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586-300x206.png 300w\" sizes=\"auto, (max-width: 392px) 100vw, 392px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Click on <a href=\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/creation-of-bucket\/\" target=\"_blank\" rel=\"noreferrer noopener\" data-type=\"URL\" data-id=\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/creation-of-bucket\/\">Create Bucket<\/a><\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"372\" height=\"180\" class=\"wp-image-9310\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-587.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-587.png 372w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-587-300x145.png 300w\" sizes=\"auto, (max-width: 372px) 100vw, 372px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Create one bucket with same name as the project ID. Click create<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"497\" height=\"504\" class=\"wp-image-9311\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-588.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-588.png 497w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-588-296x300.png 296w\" sizes=\"auto, (max-width: 497px) 100vw, 497px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>The bucket will be created.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"492\" height=\"277\" class=\"wp-image-9312\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-589.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-589.png 492w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-589-300x169.png 300w\" sizes=\"auto, (max-width: 492px) 100vw, 492px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>In <a href=\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/creating-cloud-composer\/\" target=\"_blank\" rel=\"noreferrer noopener\" data-type=\"URL\" data-id=\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/creating-cloud-composer\/\">Composer<\/a>, Click on Airflow<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"733\" height=\"187\" class=\"wp-image-9313\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-590.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-590.png 733w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-590-300x77.png 300w\" sizes=\"auto, (max-width: 733px) 100vw, 733px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Choose the login<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"472\" height=\"524\" class=\"wp-image-9314\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-591.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-591.png 472w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-591-270x300.png 270w\" sizes=\"auto, (max-width: 472px) 100vw, 472px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>DAG Airflow will be opened.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"208\" class=\"wp-image-9315\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-592.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-592.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-592-300x84.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Go to Menu &gt; <a href=\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/creating-standard-gke-cluster\/\" target=\"_blank\" rel=\"noreferrer noopener\" data-type=\"URL\" data-id=\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/creating-standard-gke-cluster\/\">Kubernetes Engine <\/a>&gt; Clusters<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"447\" height=\"255\" class=\"wp-image-9316\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-593.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-593.png 447w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-593-300x171.png 300w\" sizes=\"auto, (max-width: 447px) 100vw, 447px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>The cluster has been created.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"172\" class=\"wp-image-9317\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-594.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-594.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-594-300x70.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>In Airflow, Go to Admin &gt; Variables<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"734\" height=\"272\" class=\"wp-image-9318\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-595.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-595.png 734w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-595-300x111.png 300w\" sizes=\"auto, (max-width: 734px) 100vw, 734px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Click on Create.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"295\" class=\"wp-image-9319\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-596.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-596.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-596-300x119.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p><strong>Key\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0Val<\/strong><\/p>\r\n\r\n\r\n\r\n<p>gcp_project\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 &lt;project-ID&gt;<\/p>\r\n\r\n\r\n\r\n<p>gcs_bucket\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 gs:\/\/&lt;bucket-name&gt;<\/p>\r\n\r\n\r\n\r\n<p>gce_zone\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 &lt;zone of cluster&gt;<\/p>\r\n\r\n\r\n\r\n<p>Do these one by one in Key and Val. And press Save and Add Another<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"593\" class=\"wp-image-9320\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-597.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-597.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-597-300x240.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"606\" class=\"wp-image-9321\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-598.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-598.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-598-300x245.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>In last one Press save<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"598\" class=\"wp-image-9322\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-599.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-599.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-599-300x242.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>The key and Value will be added.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"215\" class=\"wp-image-9323\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-600.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-600.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-600-300x87.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Open Composer.<\/p>\r\n\r\n\r\n\r\n<p>Click on DAGs Folder<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"159\" class=\"wp-image-9324\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-601.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-601.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-601-300x64.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Copy the path.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"630\" height=\"279\" class=\"wp-image-9325\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-602.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-602.png 630w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-602-300x133.png 300w\" sizes=\"auto, (max-width: 630px) 100vw, 630px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Click on Activate Cloud Shell<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"382\" height=\"139\" class=\"wp-image-9326\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-603.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-603.png 382w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-603-300x109.png 300w\" sizes=\"auto, (max-width: 382px) 100vw, 382px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Paste the below code in shell. In DAG path paste the copied DAG path and press Enter<\/p>\r\n\r\n\r\n\r\n<p>$\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 gsutil cp gs:\/\/cloud-training\/datawarehousing\/lab_assets\/hadoop_tutorial.py gs:\/\/&lt;paste the DAG path&gt;<\/p>\r\n\r\n\r\n\r\n<p>It will copy hadoop_tutorial.py\u00a0 into Cluster bucket<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"127\" class=\"wp-image-9327\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-604.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-604.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-604-300x51.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<div class=\"flex-1 overflow-hidden\">\r\n<div class=\"react-scroll-to-bottom--css-frkrv-79elbk h-full\">\r\n<div class=\"react-scroll-to-bottom--css-frkrv-1n7m0yu\">\r\n<div class=\"flex flex-col text-sm pb-9\">\r\n<div class=\"w-full text-token-text-primary\" data-testid=\"conversation-turn-23\">\r\n<div class=\"px-4 py-2 justify-center text-base md:gap-6 m-auto\">\r\n<div class=\"flex flex-1 text-base mx-auto gap-3 md:px-5 lg:px-1 xl:px-5 md:max-w-3xl lg:max-w-[40rem] xl:max-w-[48rem] group final-completion\">\r\n<div class=\"relative flex w-full flex-col agent-turn\">\r\n<div class=\"flex-col gap-1 md:gap-3\">\r\n<div class=\"flex flex-grow flex-col max-w-full\">\r\n<div class=\"min-h-[20px] text-message flex flex-col items-start gap-3 whitespace-pre-wrap break-words [.text-message+&amp;]:mt-5 overflow-x-auto\" data-message-author-role=\"assistant\" data-message-id=\"2c3c081e-cf49-4d3e-b614-c9f2c5ca60a8\">\r\n<div class=\"markdown prose w-full break-words dark:prose-invert light\">\r\n<p>The file will be copied into the cluster bucket, and it will also be displayed in Airflow.<\/p>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"285\" class=\"wp-image-9328\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-605.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-605.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-605-300x115.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>In Airflow, click on the DAG.\u00a0<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"147\" class=\"wp-image-9329\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-606.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-606.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-606-300x60.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<div class=\"flex-1 overflow-hidden\">\r\n<div class=\"react-scroll-to-bottom--css-frkrv-79elbk h-full\">\r\n<div class=\"react-scroll-to-bottom--css-frkrv-1n7m0yu\">\r\n<div class=\"flex flex-col text-sm pb-9\">\r\n<div class=\"w-full text-token-text-primary\" data-testid=\"conversation-turn-25\">\r\n<div class=\"px-4 py-2 justify-center text-base md:gap-6 m-auto\">\r\n<div class=\"flex flex-1 text-base mx-auto gap-3 md:px-5 lg:px-1 xl:px-5 md:max-w-3xl lg:max-w-[40rem] xl:max-w-[48rem] group final-completion\">\r\n<div class=\"relative flex w-full flex-col agent-turn\">\r\n<div class=\"flex-col gap-1 md:gap-3\">\r\n<div class=\"flex flex-grow flex-col max-w-full\">\r\n<div class=\"min-h-[20px] text-message flex flex-col items-start gap-3 whitespace-pre-wrap break-words [.text-message+&amp;]:mt-5 overflow-x-auto\" data-message-author-role=\"assistant\" data-message-id=\"3b6bacbb-1f7e-45c1-9ea9-fcd25140a940\">\r\n<div class=\"markdown prose w-full break-words dark:prose-invert light\">\r\n<p>Please click on the Graph View. You will see a graph-like structure.<\/p>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"380\" class=\"wp-image-9330\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-607.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-607.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-607-300x154.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Hover the curser to each one. You can see the details.<\/p>\r\n\r\n\r\n\r\n<p>Click any one of it.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"582\" height=\"246\" class=\"wp-image-9331\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-608.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-608.png 582w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-608-300x127.png 300w\" sizes=\"auto, (max-width: 582px) 100vw, 582px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Press View Log.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"546\" height=\"478\" class=\"wp-image-9332\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-609.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-609.png 546w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-609-300x263.png 300w\" sizes=\"auto, (max-width: 546px) 100vw, 546px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>You can see the log for the Execution.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"270\" class=\"wp-image-9333\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-610.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-610.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-610-300x109.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<h3>Go to Bucket. Open the Bucket which we created.<\/h3>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"501\" height=\"280\" class=\"wp-image-9334\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-611.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-611.png 501w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-611-300x168.png 300w\" sizes=\"auto, (max-width: 501px) 100vw, 501px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>file saved.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"656\" height=\"288\" class=\"wp-image-9335\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-612.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-612.png 656w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-612-300x132.png 300w\" sizes=\"auto, (max-width: 656px) 100vw, 656px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>If it is not Executed, Open the Airflow &gt; composer_hadoop_tutorial.<\/p>\r\n\r\n\r\n\r\n<p>\u00a0Trigger DAG<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"277\" class=\"wp-image-9336\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-613.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-613.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-613-300x112.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Trigger<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"562\" height=\"226\" class=\"wp-image-9337\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-614.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-614.png 562w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-614-300x121.png 300w\" sizes=\"auto, (max-width: 562px) 100vw, 562px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>\u00a0Graph view. Here you can see the execution.<\/p>\r\n\r\n\r\n\r\n<p>The below colors shows the execution state.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"663\" height=\"59\" class=\"wp-image-9338\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-615.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-615.png 663w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-615-300x27.png 300w\" sizes=\"auto, (max-width: 663px) 100vw, 663px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Its runnning create_dataproc_cluster.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"344\" class=\"wp-image-9339\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-616.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-616.png 600w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-616-300x172.png 300w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Open Menu &gt; Dataproc &gt; Clusters<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"456\" height=\"339\" class=\"wp-image-9340\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-617.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-617.png 456w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-617-300x223.png 300w\" sizes=\"auto, (max-width: 456px) 100vw, 456px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>cluster\u00a0 created.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"186\" class=\"wp-image-9341\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-618.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-618.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-618-300x75.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Now the green border is on run_dataproc_hadoop. It is executing the content<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"646\" height=\"332\" class=\"wp-image-9342\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-619.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-619.png 646w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-619-300x154.png 300w\" sizes=\"auto, (max-width: 646px) 100vw, 646px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Then it changes to delete_dataproc_cluster. It will delete the cluster.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"591\" height=\"340\" class=\"wp-image-9343\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-620.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-620.png 591w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-620-300x173.png 300w\" sizes=\"auto, (max-width: 591px) 100vw, 591px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Check the cluster in dataproc. deleted.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"587\" height=\"199\" class=\"wp-image-9344\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-621.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-621.png 587w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-621-300x102.png 300w\" sizes=\"auto, (max-width: 587px) 100vw, 587px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Open the Dataproc &gt; jobs. Open the job.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"592\" height=\"186\" class=\"wp-image-9345\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-622.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-622.png 592w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-622-300x94.png 300w\" sizes=\"auto, (max-width: 592px) 100vw, 592px\" \/><\/figure>\r\n\r\n\r\n\r\n<div class=\"flex-1 overflow-hidden\">\r\n<div class=\"react-scroll-to-bottom--css-frkrv-79elbk h-full\">\r\n<div class=\"react-scroll-to-bottom--css-frkrv-1n7m0yu\">\r\n<div class=\"flex flex-col text-sm pb-9\">\r\n<div class=\"w-full text-token-text-primary\" data-testid=\"conversation-turn-29\">\r\n<div class=\"px-4 py-2 justify-center text-base md:gap-6 m-auto\">\r\n<div class=\"flex flex-1 text-base mx-auto gap-3 md:px-5 lg:px-1 xl:px-5 md:max-w-3xl lg:max-w-[40rem] xl:max-w-[48rem] group final-completion\">\r\n<div class=\"relative flex w-full flex-col agent-turn\">\r\n<div class=\"flex-col gap-1 md:gap-3\">\r\n<div class=\"flex flex-grow flex-col max-w-full\">\r\n<div class=\"min-h-[20px] text-message flex flex-col items-start gap-3 whitespace-pre-wrap break-words [.text-message+&amp;]:mt-5 overflow-x-auto\" data-message-author-role=\"assistant\" data-message-id=\"dca5ba9c-a459-4e69-af46-777103272e05\">\r\n<div class=\"markdown prose w-full break-words dark:prose-invert light\">\r\n<p>\u00a0file saved.<\/p>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"605\" height=\"743\" class=\"wp-image-9346\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-623.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-623.png 605w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-623-244x300.png 244w\" sizes=\"auto, (max-width: 605px) 100vw, 605px\" \/><\/figure>\r\n\r\n\r\n\r\n<div class=\"flex-1 overflow-hidden\">\r\n<div class=\"react-scroll-to-bottom--css-frkrv-79elbk h-full\">\r\n<div class=\"react-scroll-to-bottom--css-frkrv-1n7m0yu\">\r\n<div class=\"flex flex-col text-sm pb-9\">\r\n<div class=\"w-full text-token-text-primary\" data-testid=\"conversation-turn-27\">\r\n<div class=\"px-4 py-2 justify-center text-base md:gap-6 m-auto\">\r\n<div class=\"flex flex-1 text-base mx-auto gap-3 md:px-5 lg:px-1 xl:px-5 md:max-w-3xl lg:max-w-[40rem] xl:max-w-[48rem] group final-completion\">\r\n<div class=\"relative flex w-full flex-col agent-turn\">\r\n<div class=\"flex-col gap-1 md:gap-3\">\r\n<div class=\"flex flex-grow flex-col max-w-full\">\r\n<div class=\"min-h-[20px] text-message flex flex-col items-start gap-3 whitespace-pre-wrap break-words [.text-message+&amp;]:mt-5 overflow-x-auto\" data-message-author-role=\"assistant\" data-message-id=\"1469b91f-430a-4d35-a4f8-1e27ba43ad36\">\r\n<div class=\"markdown prose w-full break-words dark:prose-invert light\">\r\n<p>The system will save the file in the bucket.<\/p>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n<\/div>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"656\" height=\"288\" class=\"wp-image-9347\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-624.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-624.png 656w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-624-300x132.png 300w\" sizes=\"auto, (max-width: 656px) 100vw, 656px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>In Airflow click on Code.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"277\" class=\"wp-image-9348\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-625.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-625.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-625-300x112.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n<p>&nbsp;<\/p>\r\n\r\n\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"703\" class=\"wp-image-9349\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-626.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-626.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-626-300x285.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>To delete the composer environment , Click on Delete.<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"141\" class=\"wp-image-9350\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-627.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-627.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-627-300x57.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n\r\n\r\n\r\n<p>Press Delete<\/p>\r\n\r\n\r\n\r\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"741\" height=\"229\" class=\"wp-image-9351\" src=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-628.png\" alt=\"\" srcset=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-628.png 741w, https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-628-300x93.png 300w\" sizes=\"auto, (max-width: 741px) 100vw, 741px\" \/><\/figure>\r\n<p>&nbsp;<\/p>\r\n<p><span data-sheets-root=\"1\" data-sheets-value=\"{&quot;1&quot;:2,&quot;2&quot;:&quot;Cloud Composer and Airflow Integration&quot;}\" data-sheets-userformat=\"{&quot;2&quot;:513,&quot;3&quot;:{&quot;1&quot;:0},&quot;12&quot;:0}\">Cloud Composer and Airflow Integration<\/span><\/p>\r\n","protected":false},"excerpt":{"rendered":"<p>Cloud Composer and Airflow Integration Prerequisites GCP account Open Console Open Menu &gt; Cloud Storage &gt; Browser Click on Create Bucket Create one bucket with same name as the project ID. Click create The bucket will be created. In Composer, Click on Airflow Choose the login DAG Airflow will be opened. Go to Menu &gt; [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1632,1],"tags":[1519,1520,1521,1522,605,699,700,617,683,684,685,611,1400,686,692],"class_list":["post-9304","post","type-post","status-publish","format-standard","hentry","category-bigquery","category-google-cloud-platform","tag-cloud-composer","tag-cloud-composer-dag","tag-cloud-composer-documentation","tag-cloud-composer-environment","tag-gcp","tag-gcp-certification","tag-gcp-cloud-console","tag-google-cloud","tag-google-cloud-certification","tag-google-cloud-console","tag-google-cloud-courses","tag-google-cloud-platform","tag-google-cloud-platform-tutorial","tag-google-cloud-storage","tag-google-cloud-training"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v25.7 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Cloud Composer and Airflow Integration - Prwatech<\/title>\n<meta name=\"description\" content=\"Master Cloud Composer and Airflow Integration with Prwatech - Dive deep with our expert instructors and comprehensive curriculum.\" \/>\n<meta name=\"robots\" content=\"noindex, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Cloud Composer and Airflow Integration - Prwatech\" \/>\n<meta property=\"og:description\" content=\"Master Cloud Composer and Airflow Integration with Prwatech - Dive deep with our expert instructors and comprehensive curriculum.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/\" \/>\n<meta property=\"og:site_name\" content=\"Prwatech\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/prwatech.in\/\" \/>\n<meta property=\"article:published_time\" content=\"2021-05-31T05:31:21+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-03-19T09:35:01+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png\" \/>\n<meta name=\"author\" content=\"Prwatech\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@Eduprwatech\" \/>\n<meta name=\"twitter:site\" content=\"@Eduprwatech\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Prwatech\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/\",\"url\":\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/\",\"name\":\"Cloud Composer and Airflow Integration - Prwatech\",\"isPartOf\":{\"@id\":\"https:\/\/prwatech.in\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png\",\"datePublished\":\"2021-05-31T05:31:21+00:00\",\"dateModified\":\"2024-03-19T09:35:01+00:00\",\"author\":{\"@id\":\"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3\"},\"description\":\"Master Cloud Composer and Airflow Integration with Prwatech - Dive deep with our expert instructors and comprehensive curriculum.\",\"breadcrumb\":{\"@id\":\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#primaryimage\",\"url\":\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png\",\"contentUrl\":\"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png\",\"width\":392,\"height\":269},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/prwatech.in\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Working with composer using airflow\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/prwatech.in\/blog\/#website\",\"url\":\"https:\/\/prwatech.in\/blog\/\",\"name\":\"Prwatech\",\"description\":\"Share Ideas, Start Something Good.\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/prwatech.in\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3\",\"name\":\"Prwatech\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/prwatech.in\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g\",\"caption\":\"Prwatech\"},\"url\":\"https:\/\/prwatech.in\/blog\/author\/prwatech123\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Cloud Composer and Airflow Integration - Prwatech","description":"Master Cloud Composer and Airflow Integration with Prwatech - Dive deep with our expert instructors and comprehensive curriculum.","robots":{"index":"noindex","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"og_locale":"en_US","og_type":"article","og_title":"Cloud Composer and Airflow Integration - Prwatech","og_description":"Master Cloud Composer and Airflow Integration with Prwatech - Dive deep with our expert instructors and comprehensive curriculum.","og_url":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/","og_site_name":"Prwatech","article_publisher":"https:\/\/www.facebook.com\/prwatech.in\/","article_published_time":"2021-05-31T05:31:21+00:00","article_modified_time":"2024-03-19T09:35:01+00:00","og_image":[{"url":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png","type":"","width":"","height":""}],"author":"Prwatech","twitter_card":"summary_large_image","twitter_creator":"@Eduprwatech","twitter_site":"@Eduprwatech","twitter_misc":{"Written by":"Prwatech","Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/","url":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/","name":"Cloud Composer and Airflow Integration - Prwatech","isPartOf":{"@id":"https:\/\/prwatech.in\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#primaryimage"},"image":{"@id":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#primaryimage"},"thumbnailUrl":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png","datePublished":"2021-05-31T05:31:21+00:00","dateModified":"2024-03-19T09:35:01+00:00","author":{"@id":"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3"},"description":"Master Cloud Composer and Airflow Integration with Prwatech - Dive deep with our expert instructors and comprehensive curriculum.","breadcrumb":{"@id":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#primaryimage","url":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png","contentUrl":"https:\/\/prwatech.in\/blog\/wp-content\/uploads\/2021\/05\/image-586.png","width":392,"height":269},{"@type":"BreadcrumbList","@id":"https:\/\/prwatech.in\/blog\/google-cloud-platform\/bigquery\/working-with-composer-using-airflow\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/prwatech.in\/blog\/"},{"@type":"ListItem","position":2,"name":"Working with composer using airflow"}]},{"@type":"WebSite","@id":"https:\/\/prwatech.in\/blog\/#website","url":"https:\/\/prwatech.in\/blog\/","name":"Prwatech","description":"Share Ideas, Start Something Good.","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/prwatech.in\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Person","@id":"https:\/\/prwatech.in\/blog\/#\/schema\/person\/db90baff7744090b2288bbc98fea87f3","name":"Prwatech","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/prwatech.in\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/c00bafc1b04045f31eda917de39891456c44fa47c092b9bb6be0f860a3a30a2f?s=96&d=mm&r=g","caption":"Prwatech"},"url":"https:\/\/prwatech.in\/blog\/author\/prwatech123\/"}]}},"_links":{"self":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts\/9304","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/comments?post=9304"}],"version-history":[{"count":4,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts\/9304\/revisions"}],"predecessor-version":[{"id":10936,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/posts\/9304\/revisions\/10936"}],"wp:attachment":[{"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/media?parent=9304"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/categories?post=9304"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/prwatech.in\/blog\/wp-json\/wp\/v2\/tags?post=9304"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}