Actions: Deploy Serverless Python Functions

Deploy serverless Python functions that execute on-demand via API endpoints or as tools for AI agents. Write custom logic with the @action decorator, use the context API for logging, and automate workflows like sending notifications, processing data, or calling external APIs.

Datazone Agents: Custom AI Assistants for Your Data

Create custom AI assistants that interact with your data through natural language. Agents can execute SQL queries, run Python code, generate charts, and search the web automatically. Supports multiple languages and remembers conversation context for intelligent, multi-step analysis.

Key Features:

• Model Accounts: Securely manage AI provider credentials for OpenAI, Anthropic (Claude), and AWS Bedrock. Configure once and reuse across projects and agents with encrypted storage and organization-level access control.

• New Clear Layout: Redesigned interface with card-based home page, streamlined project workspace, and ⌘K quick search for instant navigation to any resource from anywhere in the app.

Enhanced Visualization Capabilities

We're excited to bring powerful new visualization and customization options to Intelligent Apps!

Key Features:

• Scatter Plot Chart Type: Introduced a new scatter plot chart type for visualizing relationships between two numerical variables. Each point represents a data record positioned according to two metrics on the x and y axes, ideal for identifying correlations, patterns, or outliers in your data.

• Custom Theme Styling: Added custom_style_attributes for advanced theme customization. Override specific theme colors using Tailwind CSS variables in HSL format, giving you fine-grained control over backgrounds, foregrounds, and chart color palettes.

• Conditional Chart Visibility: Added hide_expression to chart_config, allowing you to conditionally hide charts based on query results. Use JavaScript expressions to control chart visibility dynamically (e.g., hide charts with insufficient data).

Policy System: Comprehensive Role-Based Access Control

Introduced a comprehensive role-based access control system with hierarchical permissions! Define fine-grained access policies with support for flat and hierarchical resource patterns (e.g., project:<id>:*), explicit allow/deny rules, and resource-specific constraints.

Key Features:

• Policies support wildcards for flexible matching
• Branch-aware permissions
• Extra constraints like row-level security for views and path restrictions for projects
• Bind policies to roles for scalable permission management across your organization

Why does this matter? The Policy System gives you enterprise-grade security and access control, ensuring that team members can only access the data and resources they're authorized to use.

Views: Optimized Relational Database Structures

Introduced a powerful new feature - Views! Transform your datasets into optimized relational database structures with advanced configurations like partitioning, primary keys, and ordering.

Key Features:

• Choose between materialized views for lightning-fast query performance or non-materialized views for real-time data access
• Create views by replicating datasets or writing custom SQL queries that combine multiple datasets and views
• Access your views through SQL interface, endpoints, and intelligent apps for enhanced performance and flexibility

Why does this matter? Views provide the performance and flexibility of traditional database views while maintaining the benefits of Datazone's lakehouse architecture, giving you the best of both worlds.

Heatmap Charts, Markdown Components, and Multiple Selection Filters

We're excited to bring new visualization and interaction capabilities to Intelligent Apps!

Key Features:

• Heatmap Chart Type: Introduced a new heatmap chart type for visualizing data where values are represented as colors. Heatmaps require exactly two dimensions and one metric, making them perfect for showing relationships between categorical dimensions.

• Markdown Component: Added support for Markdown components in Intelligent Apps. Users can now add rich text sections with formatting, headers, lists, and more to their dashboards.

• Multiple Selection Filters: Enhanced dropdown filters with multiple selection capability. Users can now select multiple values from a dropdown filter, with proper handling in SQL queries using Jinja templates.

Why does this matter? These features enhance the interactivity and visual appeal of your Intelligent Apps, making it easier to create comprehensive, user-friendly dashboards.

Embedded Intelligent Apps: More Customization and Export Options

We’re excited to bring new enhancements to Embedded Intelligent Apps in Datazone!

Key Features:

• Embed Apps via iframes: Seamlessly integrate Intelligent Apps into external applications using iframes. Generate secure JWT tokens containing app and user details to ensure safe integration.
• Theme Selection: Customize the look and feel of your Intelligent Apps with multiple color schemes: default, teal, blue, green, purple, orange, amber, and mono. Easily set themes using the style.theme configuration property.
• Chart Export Features: Users can now export chart data as CSV files or save visualizations as PNG images. Enable export capabilities with the chart_export_enabled configuration property.

Why does this matter? These updates give you full control over how your embedded apps look and function, while making it easier to share data insights with your users across platforms.

Start enhancing your Intelligent Apps today with customization and export options to create richer, more interactive experiences.

File Container & Orion Notebook Assistant Enhancements

We’re excited to share two powerful new capabilities in Datazone!

File Container: Manage Files with Ease

• Create, update, and delete file containers directly in Datazone
• Store and organize data for your pipelines and notebooks
• Interact with file containers just like AWS S3 or Google Cloud Storage buckets
• Use the FileContainerClient toolkit for seamless programmatic access

Orion Notebook Assistant: Smarter Notebooks

• Get instant code suggestions and error fixes while working in Orion Notebooks
• Simply click the ✨ Sparkles button and describe what you want
• Orion will provide ready-to-use code snippets and guidance

Why does this matter? These updates simplify data management and accelerate productivity. File Container gives you cloud-native storage inside Datazone, while Orion Assistant helps you code faster and smarter.

Ready to supercharge your workflow with these new features? Dive in today and see how Datazone makes data work easier.

Introducing Datazone Endpoints: Secure API Interfaces

We’re thrilled to announce a brand-new core concept in Datazone: Endpoints!

Key Features of Endpoints:

• Create secure, controlled API interfaces for your datasets
• Define endpoints using YAML configuration with filters, queries, and authentication
• Ideal for exposing analytical results, feeding data to downstream systems, or creating data products for other teams

Why does this matter? Endpoints provide a safe, efficient, and flexible way to share your data across teams and systems, enabling better collaboration and faster access to insights.

Start using Endpoints today to unlock the full potential of your Datazone datasets.

Datazone Now Supports S/4HANA Cloud Public Edition Integration

We are thrilled to announce that Datazone now offers seamless integration with S/4HANA Cloud Public Edition!

This game-changing capability empowers S/4HANA Cloud customers to:

• To have analytic capabilities in just minutes not months without breaking the bank
• Access your S/4HANA Cloud Public Edition data through Datazone-SQL without impacting your core business systems
• Unlock your data for advanced reporting tools and AI applications
• Enable secure external data consumption while preserving system performance
• Leverage Datazone's robust datalakehouse architecture for all analytical workloads

Why does this matter? Your business-critical S/4HANA Cloud system stays focused on what it does best, while Datazone handles the heavy lifting of analytics, reporting, and AI integration.

Ready to maximize the value of your S/4HANA Cloud data without compromise? Let's connect to discuss how Datazone can transform your data strategy.

Datazone Now Supports Pandas Engine!

We're thrilled to announce that Datazone has expanded its capabilities with native Pandas support!

While our Apache Spark foundation continues to power your large-scale distributed data workloads, our new pandas option brings exceptional efficiency to lightweight pipelines where distributed execution might be overkill.

Why This Matters:

• Performance Boost: Dramatically faster execution for smaller datasets and simpler transformations
• Resource Optimization: Right-size your compute resources based on workload requirements
• Simplified Development: Leverage pandas' intuitive API for rapid data manipulation

This enhancement represents our commitment to providing the right tool for every job within our comprehensive data management and AI platform.

Whether you're processing terabytes of data across a cluster or performing agile analytics on targeted datasets , Datazone now offers the perfect engine for your specific needs.

Datazone now includes built-in CI/CD Checks

Datazone now includes built-in CI/CD checks to ensure your data pipelines are production-ready! Run automated tests, validate data quality, and verify schema compatibility before deployment. Our new integration helps catch issues early, maintain code standards, and deploy with confidence.

Features include code quality checks, data validation testing, and automated regression testing - everything you need to maintain reliable data pipelines.

Orion AI Assistant Now Builds Complete Data Pipelines

We're excited to share a major enhancement to Orion, our AI assistant! Orion now goes beyond helping with individual transforms - it can automatically generate entire data pipelines through natural language commands.

New Pipeline Generation Features

• Convert end-to-end pipeline requirements into connected PySpark transforms
• Automatically handle dependencies between transforms
• Generate proper input and output mappings across the pipeline
• Create optimized execution flows
• Maintain all existing Orion capabilities you know and love

This powerful new capability means you can now describe your complete data workflow to Orion, and it will generate all the necessary transforms and connect them into a production-ready pipeline.

Try Orion's new pipeline generation capabilities today and experience an even more streamlined way to build your data workflows.

Introducing Datazone Public Edition

We're thrilled to announce the launch of Datazone Public Edition - bringing enterprise-grade data analytics capabilities to everyone in the data engineering community!

What's Included

• Complete data journey management from ingestion to serving
• 300+ pre-built connectors
• Built-in code and data versioning
• Powered by Apache Spark and ClickHouse
• Advanced features like data time travel and branching

With Datazone Public Edition, we're democratizing data analytics by making our comprehensive platform accessible to individual engineers, smaller teams, and organizations looking to streamline their data operations.

Start transforming your data journey today with Datazone Public Edition - where powerful data analytics meets simplicity.

Introducing Datazone Airflow Operator

We're thrilled to announce the release of the Datazone Airflow Operator , a powerful new tool designed to bridge the gap between Datazone and Apache Airflow. This integration empowers existing Airflow users to seamlessly orchestrate Datazone tasks within their familiar Airflow environment.

Key Features and Benefits:

• Effortless Integration: Easily incorporate Datazone tasks into your existing Airflow workflows, allowing for a smooth transition and coexistence of technologies.
• Best of Both Worlds: Leverage Datazone's robust data management capabilities alongside Airflow's scheduling and orchestration features.
• Familiar Interface: Airflow users can continue working in their preferred environment while taking advantage of Datazone's advanced features.
• Simplified Migration: Gradually adopt Datazone features without the need for an immediate, full-scale migration of your existing workflows.
• Enhanced Monitoring: Monitor Datazone tasks directly from your Airflow dashboard, providing a unified view of your entire data pipeline.
• Customizable Task Parameters: Easily configure Datazone task parameters directly within your Airflow DAGs.

This new operator underscores our commitment to flexibility and interoperability. While Datazone offers powerful orchestration features of its own, we understand the importance of supporting diverse infrastructures and workflows. The Datazone Airflow Operator is designed to make your transition to Datazone smooth and tailored to your needs.

Introducing Data Projects: Where Data Magic happens

We are excited to unveil the latest addition to the Datazone platform - Data Projects. This powerful new feature serves as the central repository for all aspects of your data journey, from ingestion to monitoring, streamlining your data management processes like never before.

Key Features of Data Projects:

• Unified Workspace: Data Projects provide a single, cohesive environment for data ingestion, modeling, code authoring, execution, and monitoring. No more switching between different tools or interfaces - everything you need is in one place.
• End-to-End Data Journey Support: Aligned with Datazone's comprehensive approach, Data Projects cover every step of the data journey:
  - Data Ingestion
  - Data Transformation
  - Data Storage and Versioning
  - Data Analysis and Exploration
  - Data Branching
  - Training and Inference
  - Serving Data to the outside world
• Collaborative Environment: Foster teamwork with built-in collaboration tools. Share projects, code, and insights seamlessly with your team members, enhancing productivity and knowledge sharing.
• Version Control: Keep track of changes with integrated version control for both code and data. This feature allows you to maintain a clear history of your project's evolution and easily revert to previous states if needed.
• Integrated Monitoring: Track the health and performance of your data pipelines in real-time. Set up alerts and gain insights into your data processes without leaving the Data Projects interface.
• Scalability: Like all Datazone features, Data Projects are designed to scale with your needs. Whether you're working with gigabytes or petabytes of data, Data Projects can handle it efficiently.
• Security and Compliance: Enjoy peace of mind with Datazone's robust security measures. Data Projects adhere to the same high standards of data protection and compliance that you expect from Datazone.

Introducing Notebooks : Your freedom of Analysis

We are thrilled to announce the latest enhancement to the Datazone platform - Datazone Notebooks. This new feature brings the power and flexibility of Jupyter-like notebooks directly into your Datazone environment, enabling a more dynamic and interactive way to work with your data.

Key Features of Datazone Notebooks:

• Interactive Coding Environment: Execute code in Python, SQL, or natural language codes directly within your browser. Datazone Notebooks support real-time execution, allowing you to see the results instantly as you refine your data analysisscripts.
• Seamless Integration: Datazone Notebooks are fully integrated with all other Datazone functionalities. This means you can easily pull data from your existing Datazone pipelines, analyze it in the notebook.
• Enhanced Security: As with all Datazone features, the notebooks are built with security in mind. Your data and code are protected under Datazone's comprehensive security framework.

Introducing Datazone Orion: Your AI-Powered Coding Companion

We're thrilled to unveil an extraordinary addition to the Datazone suite - Orion, an AI agent crafted to transform the way data professionals work. Designed specifically for data engineers, data scientists, data analysts, and machine learning engineers, Orion is here to redefine coding efficiency and intelligence.

Orion at a Glance:

• English to Code Translation: Say goodbye to the barriers between idea and implementation. With Orion, just describe your coding objectives in plain English, and watch as it translates your thoughts into executable code.
• Error Analysis and Correction Suggestions: Orion doesn't just identify errors; it understands them. It offers insightful corrections and suggestions, making troubleshooting a breeze.
• A Partner in Code: Whether you're drafting new algorithms or refining existing ones, Orion acts as your coding partner, offering guidance, suggestions, and insights to enhance your work.

Datazone is now multicloud!

We're excited to announce a significant milestone in Datazone's journey—our platform is now fully operational across multiple cloud providers, including AWS, Azure, and Google Cloud Platform (GCP)! This multi-cloud capability ensures that Datazone offers unparalleled flexibility, resilience, and efficiency, allowing you to leverage the unique strengths of each cloud service. This update underscores our commitment to providing you with a robust, scalable, and versatile data management solution. Welcome to a new era of cloud-enabled data possibilities with Datazone!

Datazone Version 1.0 Release

We are thrilled to announce the release of Datazone version 1.0! This release represents a significant milestone in our journey to provide you with the best data management and AI experience.