Free Databricks Generative AI Engineer Associate practice questions

10 free Databricks Generative AI Engineer Associate practice questions with the correct answer and a full explanation for each, taken from the CertStash pack of 109 questions. Work through them, then open each answer to check your reasoning.

Get all 109 questions (US$39) · Download these 10 as a PDF

Question 1

A Generative Al Engineer has created a RAG application to look up answers to questions about a series of fantasy novels that are being asked on the author’s web forum. The fantasy novel texts are chunked and embedded into a vector store with metadata (page number, chapter number, book title), retrieved with the user’s query, and provided to an LLM for response generation. The Generative AI Engineer used their intuition to pick the chunking strategy and associated configurations but now wants to more methodically choose the best values.

Which TWO strategies should the Generative AI Engineer take to optimize their chunking strategy and parameters? (Choose two.)

  1. Change embedding models and compare performance.
  2. Add a classifier for user queries that predicts which book will best contain the answer. Use this to filter retrieval.
  3. Choose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the chunking strategy, such as splitting chunks by paragraphs or chapters. Choose the strategy that gives the best performance metric.
  4. Pass known questions and best answers to an LLM and instruct the LLM to provide the best token count. Use a summary statistic (mean, median, etc.) of the best token counts to choose chunk size.
  5. Create an LLM-as-a-judge metric to evaluate how well previous questions are answered by the most appropriate chunk. Optimize the chunking parameters based upon the values of the metric.
Show answer and explanation

Correct answer: C, E

C. Choose an appropriate evaluation metric (such as recall or NDCG) and experiment with changes in the chunking strategy, such as splitting chunks by paragraphs or chapters. Choose the strategy that gives the best performance metric. E. Create an LLM-as-a-judge metric to evaluate how well previous questions are answered by the most appropriate chunk. Optimize the chunking parameters based upon the values of the metric. Option C is correct because systematic evaluation using appropriate metrics (recall, NDCG) with experimentation on different chunking strategies (by paragraphs, chapters, token counts) provides empirical evidence for the best configuration. Option E is correct because using an LLM-as-a-judge metric directly evaluates retrieval quality by assessing how well chunks answer known questions, allowing data-driven optimization of chunking parameters. Both approaches replace intuition with measurable evaluation frameworks.

Why the other options are wrong

  • A. Changing embedding models addresses a different component and doesn't optimize the chunking strategy itself.
  • B. Adding a query classifier filters which book to search but doesn't optimize the chunking parameters.
  • D. Having an LLM predict ideal token counts lacks empirical validation and doesn't account for actual retrieval performance with real queries.

Question 2

A Generative AI Engineer is designing a RAG application for answering user questions on technical regulations as they learn a new sport.

What are the steps needed to build this RAG application and deploy it?

  1. Ingest documents from a source –> Index the documents and saves to Vector Search –> User submits queries against an LLM –> LLM retrieves relevant documents –> Evaluate model –> LLM generates a response –> Deploy it using Model Serving
  2. Ingest documents from a source –> Index the documents and save to Vector Search –> User submits queries against an LLM –> LLM retrieves relevant documents –> LLM generates a response -> Evaluate model –> Deploy it using Model Serving
  3. Ingest documents from a source –> Index the documents and save to Vector Search –> Evaluate model –> Deploy it using Model Serving
  4. User submits queries against an LLM –> Ingest documents from a source –> Index the documents and save to Vector Search –> LLM retrieves relevant documents –> LLM generates a response –> Evaluate model –> Deploy it using Model Serving
Show answer and explanation

Correct answer: B. Ingest documents from a source –> Index the documents and save to Vector Search –> User submits queries against an LLM –> LLM retrieves relevant documents –> LLM generates a response -> Evaluate model –> Deploy it using Model Serving

The correct sequence for RAG deployment is: ingest documents → index and save to vector store → user submits query → LLM retrieves relevant documents → LLM generates response → evaluate model → deploy. Option B maintains this logical flow where document preparation precedes user interaction, retrieval occurs before generation, and evaluation happens before deployment. This ensures the system is fully built and tested before production use.

Why the other options are wrong

  • A. Evaluating the model after generation but before deployment is correct, but this option shows evaluation occurring between retrieval and generation, which is premature.
  • C. This omits the critical user query submission and LLM response generation steps entirely.
  • D. Beginning with user query submission before documents are ingested and indexed is impossible since there would be no knowledge base to retrieve from.

Question 3

A Generative AI Engineer just deployed an LLM application at a digital marketing company that assists with answering customer service inquiries.

Which metric should they monitor for their customer service LLM application in production?

  1. Number of customer inquiries processed per unit of time
  2. Energy usage per query
  3. Final perplexity scores for the training of the model
  4. HuggingFace Leaderboard values for the base LLM
Show answer and explanation

Correct answer: A. Number of customer inquiries processed per unit of time

For a customer service LLM application, monitoring the number of customer inquiries processed per unit of time is essential because it directly reflects the system's utility and throughput in a production environment. This operational metric helps track whether the application is meeting business needs and handling customer load effectively.

Why the other options are wrong

  • B. Energy usage is an infrastructure concern but not a primary metric for evaluating the application's performance in serving customer inquiries.
  • C. Final perplexity scores from training are static historical metrics that don't reflect rea-ime production quality or performance.
  • D. HuggingFace Leaderboard values reflect the base model's general capabilities, not the specific application's performance in its deployed context.

Question 4

A Generative AI Engineer is building a Generative AI system that suggests the best matched employee team member to newly scoped projects.

The team member is selected from a very large team. The match should be based upon project date availability and how well their employee profile matches the project scope. Both the employee profile and project scope are unstructured text.

How should the Generative Al Engineer architect their system?

  1. Create a tool for finding available team members given project dates. Embed all project scopes into a vector store, perform a retrieval using team member profiles to find the best team member.
  2. Create a tool for finding team member availability given project dates, and another tool that uses an LLM to extract keywords from project scopes. Iterate through available team members’ profiles and perform keyword matching to find the best available team member.
  3. Create a tool to find available team members given project dates. Create a second tool that can calculate a similarity score for a combination of team member profile and the project scope. Iterate through the team members and rank by best score to select a team member.
  4. Create a tool for finding available team members given project dates. Embed team profiles into a vector store and use the project scope and filtering to perform retrieval to find the available best matched team members.
Show answer and explanation

Correct answer: D. Create a tool for finding available team members given project dates. Embed team profiles into a vector store and use the project scope and filtering to perform retrieval to find the available best matched team members.

Option D is correct because it combines both required capabilities: a tool to filter available team members by project dates, and embedding team profiles into a vector store to use the project scope for semantic similarity retrieval. This approach leverages semantic matching of unstructured text (profiles and project scope) through embeddings while respecting the availability constraint, creating an efficient system for large team sizes.

Why the other options are wrong

  • A. Embedding project scopes but retrieving with team member profiles is backwards; profiles should be embedded and queried against project scope.
  • B. Keyword matching on extracted keywords from unstructured text is brittle and less effective than semantic embedding similarity for nuanced matching.
  • C. Iterating through all available team members and calculating scores is inefficient for very large teams; vector retrieval would be much faster.

Question 5

A Generative AI Engineer is designing an LLM-powered live sports commentary platform. The platform provides real-time updates and LLM-generated analyses for any users who would like to have live summaries, rather than reading a series of potentially outdated news articles.

Which tool below will give the platform access to real-time data for generating game analyses based on the latest game scores?

  1. DatabricksIQ
  2. Foundation Model APIs
  3. Feature Serving
  4. AutoML
Show answer and explanation

Correct answer: C. Feature Serving

Feature Serving is specifically designed to serve real-time features and current data to applications. For a live sports commentary platform requiring up-to-the-moment game scores and statistics, Feature Serving provides the low-latency infrastructure to deliver real-time data that the LLM can then use for analysis and commentary generation.

Why the other options are wrong

  • A. DatabricksIQ is a natural language query tool for data exploration, not a real-time data serving mechanism.
  • B. Foundation Model APIs provide access to LLM models themselves but not specifically to real-time external data sources.
  • D. AutoML is used for model training and automation, not for accessing or serving rea-ime game data.

Question 6

A Generative AI Engineer has a provisioned throughput model serving endpoint as part of a RAG application and would like to monitor the serving endpoint’s incoming requests and outgoing responses. The current approach is to include a micro-service in between the endpoint and the user interface to write logs to a remote server.

Which Databricks feature should they use instead which will perform the same task?

  1. Vector Search
  2. Lakeview
  3. DBSQL
  4. Inference Tables
Show answer and explanation

Correct answer: D. Inference Tables

Inference Tables is the Databricks feature that automatically logs incoming requests and outgoing responses from model serving endpoints. This eliminates the need for a custom micro-service by providing built-in monitoring and logging capabilities specifically designed for this purpose in production RAG applications.

Why the other options are wrong

  • A. Vector Search is for similarity search over embeddings, not for monitoring endpoint traffic.
  • B. Lakeview is a visualization tool for creating dashboards, not an automated logging mechanism for serving endpoints.
  • C. DBSQL is for querying data in Delta Lake, not for capturing model serving endpoint logs.

Question 7

A Generative Al Engineer is tasked with improving the RAG quality by addressing its inflammatory outputs.

Which action would be most effective in mitigating the problem of offensive text outputs?

  1. Increase the frequency of upstream data updates
  2. Inform the user of the expected RAG behavior
  3. Restrict access to the data sources to a limited number of users
  4. Curate upstream data properly that includes manual review before it is fed into the RAG system
Show answer and explanation

Correct answer: D. Curate upstream data properly that includes manual review before it is fed into the RAG system

Curating upstream data with manual review before it enters the RAG system is the most effective solution because offensive outputs in RAG applications originate from problematic content in the source documents. By filtering and reviewing data at the source, you prevent inflammatory content from being embedded and retrieved in the first place, addressing the root cause.

Why the other options are wrong

  • A. Increasing update frequency doesn't address whether the data being updated contains offensive content.
  • B. Informing users about expected behavior doesn't mitigate the actual problem of inflammatory outputs being generated.
  • C. Restricting user access doesn't prevent offensive content from being generated; it merely limits who sees it.

Question 8

A Generative Al Engineer is creating an LLM-based application. The documents for its retriever have been chunked to a maximum of 512 tokens each. The Generative Al Engineer knows that cost and latency are more important than quality for this application. They have several context length levels to choose from.

Which will fulfill their need?

  1. context length 514; smallest model is 0.44GB and embedding dimension 768
  2. context length 2048: smallest model is 11GB and embedding dimension 2560
  3. context length 32768: smallest model is 14GB and embedding dimension 4096
  4. context length 512: smallest model is 0.13GB and embedding dimension 384
Show answer and explanation

Correct answer: D. context length 512: smallest model is 0.13GB and embedding dimension 384

With 512-token chunks and priorities of cost and latency over quality, option D is optimal. A context length of 512 exactly matches chunk size (no wasteful overhead), while the smallest model size of 0.13GB and lowest embedding dimension of 384 minimize both computational cost and latency. This configuration provides sufficient context without unnecessary resources.

Why the other options are wrong

  • A. Context length 514 adds minimal overhead but has a larger model (0.44GB) than option D, increasing cost and latency.
  • B. Context length 2048 wastes resources with 4x the needed context, significantly increasing model size (11GB) and latency.
  • C. Context length 32768 is excessive overkill, using the largest model (14GB) and highest embedding dimension (4096), maximizing cost and latency.

Question 9

A small and cost-conscious startup in the cancer research field wants to build a RAG application using Foundation Model APIs.

Which strategy would allow the startup to build a good-quality RAG application while being cost-conscious and able to cater to customer needs?

  1. Limit the number of relevant documents available for the RAG application to retrieve from
  2. Pick a smaller LLM that is domain-specific
  3. Limit the number of queries a customer can send per day
  4. Use the largest LLM possible because that gives the best performance for any general queries
Show answer and explanation

Correct answer: B. Pick a smaller LLM that is domain-specific

A cost-conscious startup building a RAG application should select a smaller, domai-pecific LLM rather than defaulting to the largest model available. Domain-specific smaller models are optimized for particular fields like cancer research, offering better performance-per-token efficiency and lower API costs while maintaining quality within that specialized domain. This balances cost constraints with the need for quality output tailored to the startup's specific use case.

Why the other options are wrong

  • A. Limiting document retrieval degrades RAG quality by reducing context available to the model, directly harming application effectiveness.
  • C. Imposing query limits frustrates users and doesn't address the root cost issue; it arbitrarily restricts functionality rather than optimizing efficiency.
  • D. Largest LLMs maximize costs and are unnecessary for domain-specific tasks; they waste resources on capabilities not needed for cancer research RAG applications.

Question 10

Show the case study this question is based on

A Generative Al Engineer is responsible for developing a chatbot to enable their company’s internal HelpDesk Call Center team to more quickly find related tickets and provide resolution. While creating the GenAI application work breakdown tasks for this project, they realize they need to start planning which data sources (either Unity Catalog volume or Delta table) they could choose for this application. They have collected several candidate data sources for consideration: call_rep_history: a Delta table with primary keys representative_id, call_id. This table is maintained to calculate representatives’ call resolution from fields call_duration and call start_time. transcript Volume: a Unity Catalog Volume of all recordings as a *.wav files, but also a text transcript as *.txt files. call_cust_history: a Delta table with primary keys customer_id, cal1_id. This table is maintained to calculate how much internal customers use the HelpDesk to make sure that the charge back model is consistent with actual service use. call_detail: a Delta table that includes a snapshot of all call details updated hourly. It includes root_cause and resolution fields, but those fields may be empty for calls that are still active. maintenance_schedule, a Delta table that includes a listing of both HelpDesk application outages as well as planned upcoming maintenance downtimes.

They need sources that could add context to best identify ticket root cause and resolution.

Which TWO sources do that? (Choose two.)

  1. call_cust_history
  2. maintenance_schedule
  3. call_rep_history
  4. call_detail
  5. transcript Volume
Show answer and explanation

Correct answer: D, E

D. call_detail E. transcript Volume The call_detail Delta table is essential because it directly contains root_cause and resolution fields, providing explicit context for identifying ticket root causes and resolutions despite being empty for active calls. The transcript Volume is equally valuable as it contains both audio recordings and text transcripts of actual calls, offering rich contextual information about what occurred during each interaction, customer issues, and how they were resolved. Together, these sources provide both structured data and detailed call content necessary for understanding and resolving tickets.

Why the other options are wrong

  • A. call_cust_history tracks customer usage patterns and chargeback consistency but does not provide information about root causes or resolutions.
  • B. maintenance_schedule lists outages and downtimes but does not contain context about individual ticket root causes or customer-specific resolution information.
  • C. call_rep_history focuses on representative performance metrics like call duration but lacks root cause and resolution details needed for ticket context.

That was 10 of 109.

The full Databricks Generative AI Engineer Associate pack has all 109 questions, each with the answer, the explanation and why the other options are wrong, plus a questions-only copy for timed runs. US$39, paid once, with free monthly updates and a pass-or-your-money-back guarantee.

Get the full pack