Setting up New Relic AI monitoring for a DeepSeek application involves several key steps that help you gain comprehensive visibility into your AI application's performance, quality, and cost. Here's a detailed guide on how to set it up:
Step 1: Instrument Your DeepSeek Application
To start monitoring your DeepSeek application with New Relic, you first need to instrument your application. This involves integrating New Relic's Application Performance Monitoring (APM) agents into your application. Since DeepSeek uses an OpenAI-compatible API, you can use the OpenAI SDK in your application for easier integration with New Relic[2].1. Sign Up or Log In: Begin by signing up for a New Relic account or logging in if you already have one.
2. Access Integrations & Agents: Navigate to the Integrations & Agents section in your New Relic dashboard.
3. Select OpenAI Integration: In the search bar, type OpenAI and select it. This is because DeepSeek models are compatible with OpenAI APIs.
4. Choose Your Programming Language: Select the programming language used by your application (e.g., Python, Node.js).
5. Follow Guided Onboarding: Follow the guided onboarding process provided by New Relic to instrument your application. This process will walk you through setting up the necessary configurations for AI monitoring[2].
Step 2: Access AI Monitoring
Once your application is instrumented, you can start using New Relic AI monitoring:1. Navigate to AI Monitoring: Go to All Capabilities in your New Relic dashboard and click on AI Monitoring.
2. Select Your Application: In the AI Monitoring section, under All Entities, select your DeepSeek application (e.g., "deepseek-local") to view its performance data[2].
Step 3: Evaluate Performance, Quality, and Cost
With your application set up, you can now evaluate key metrics:1. APM 360 Summary: View the APM 360 summary for insights into metrics like requests, response times, token usage, and error rates. This helps identify issues quickly[2].
2. Deep Tracing View: Click on AI Responses in the APM 360 summary to see detailed traces from user input to final response, including metadata like token counts and model information[2].
3. Model Quality Evaluation: Analyze Large Language Model (LLM) responses to identify issues such as toxicity or negativity, though this feature is currently in limited preview[2].
Step 4: Compare Models
New Relic allows you to compare different AI models to optimize performance and cost:1. Model Comparison: Use New Relic's model comparison capabilities to assess how switching between models affects your application's performance and costs. This helps in selecting the most suitable model for your needs[3][10].
Step 5: Ensure Data Privacy and Security
To protect sensitive data, you can configure drop filters:1. Create Drop Filters: Navigate to Drop Filters and create filters to exclude sensitive data (e.g., personal identifiable information) from your AI requests and responses[2].
By following these steps, you can effectively set up New Relic AI monitoring for your DeepSeek application, gaining valuable insights into its performance while ensuring data privacy and security.
Citations:
[1] https://docs.newrelic.com/install/ai-monitoring/
[2] https://newrelic.com/blog/how-to-relic/deploy-deepseek-models-locally-and-monitor-with-new-relic-ai-monitoring
[3] https://cxotoday.com/press-release/new-relic-announces-the-industrys-only-observability-integration-with-deepseek-to-accelerate-ai-adoption-and-roi/
[4] https://newrelic.com/blog/how-to-relic/ai-monitoring-for-nvidia-nim
[5] https://www.dqchannels.com/news/new-relic-introduces-observability-solution-for-deepseek-ai-monitoring-8689063
[6] https://newrelic.com/blog/how-to-relic/monitor-and-optimize-complex-web-apps-with-new-relic
[7] https://newrelic.com/blog/how-to-relic/ai-monitoring
[8] https://ecommercenews.com.au/story/new-relic-unveils-ai-monitoring-for-deepseek-applications
[9] https://docs.newrelic.com/docs/ai-monitoring/intro-to-ai-monitoring/
[10] https://newrelic.com/press-release/20250203