You want to rebuild your ML pipeline for structured data on Google Cloud. You are using PySpark to conduct data transformations at scale, but your pipelines are taking over 12 hours to run. To speed up development and pipeline run time, you want to use a serverless tool and SQL syntax. You have already moved your raw data into Cloud Storage. How should you build the pipeline on Google Cloud while meeting the speed and processing requirements?
To optimize the pipeline on Google Cloud using a serverless tool with SQL syntax, you should ingest the data into BigQuery. BigQuery is serverless and supports SQL queries, which allows you to transform the data efficiently and at scale. After performing the transformations using BigQuery SQL queries, you can write the results to a new table. This approach meets both the speed and processing requirements.
It should be D .... Data Fusion is not SQL syntax ....
Agree, BQ is the only serverless that support SQL
Needs to be D as the most suitable answer given the req's in question Datafusion is more of a no code Data transformation tool
ANS: A https://cloud.google.com/data-fusion#section-1 - Data Fusion is a serverless approach leveraging the scalability and reliability of Google services like Dataproc means Data Fusion offers the best of data integration capabilities with a lower total cost of ownership. - BigQuery is serverless and supports SQL. - Dataproc is not serverless, you have to manage clusters. - Cloud SQL is not serverless, you have to manage instances.
Data Fusion is not serverless, it create dataproc to execute the job .... I think the answer is C
Data Fusion is serverless: https://cloud.google.com/data-fusion#all-features
I think you're only viewing the sentence "A serverless approach leveraging the scalability and reliability of Google services like Dataproc means Data Fusion offers the best of data integration capabilities with a lower total cost of ownership", The sentence implies that Data Fusion leverages a serverless approach, but it does not explicitly state that Data Fusion itself is serverless. It states that Data Fusion offers the best of data integration capabilities by using a serverless approach that leverages the scalability and reliability of Google services like Dataproc. So, while Data Fusion may not be fully serverless, it is designed to take advantage of serverless capabilities through its integration with Google services.
Agree, A is correct
I was thinking B, but now I'm kind of confused that nobody voted it
Data Fusion is not in SQL syntax, so no A; Dataproc is not serverless, so no B; Passing through Cloud SQL is uselss, just go with BigQuery, so no C; D is correct
Went with D
BQ is the serverless solution
Correct answer is "D"
C,D booth can be implemented as will work but D is faster for implementation
It should be A.
Vote D
Correct option is D
I'll go with D.
D - as BQ is server less and supports SQL none of the other options match both criteria
D - BigQuery is the only serverless and SQL-syntax option.
The best approach is option D: Ingest data into BigQuery and use SQL queries for transformations. This leverages BigQuery’s serverless capabilities, efficient processing, and seamless integration with other Google Cloud services.
There's an updated version of this question in the official Google Cloud certified PMLE study guide. Option D is marked as correct
option D because needs a serveless solution and sql sintax and BigQuery offer this. Datarproc is not serverless, so B is incorrect, D is correct option.