A Generative AI Engineer is building a production-ready LLM system which replies directly to customers. The solution makes use of the Foundation Model API via provisioned throughput. They are concerned that the LLM could potentially respond in a toxic or otherwise unsafe way. They also wish to perform this with the least amount of effort.
Which approach will do this?
B. Host Llama Guard on Foundation Model API and use it to detect unsafe responses. By enabling Databricks’ built-in Llama Guard directly on your Foundation Model API endpoint, you get out-of-the-box toxicity and safety checks with zero changes to your application code. The guard runs before responses are returned, blocking or redacting unsafe content according to its policy. This approach requires the least effort compared to adding custom detection calls or regex rules, and is far more proactive than relying on user reports.