Data Science Data Engineering Copilot

Generated with sparks and insights from 7 sources

Introduction

AI Assistance: Copilot provides intelligent code completion, automates routine tasks, and supplies industry-standard code templates.
Data Integration: Works with Lakehouse tables, Power BI Datasets, and pandas/spark/fabric dataframes.
Code Generation: Generates code for data visualizations, transformations, and machine learning models.
Interactive Q&A: Users can ask questions in natural language and receive code snippets or insights.
Context Awareness: Understands the schema and metadata of dataframes and lakehouse tables.

Create a Workspace: Navigate to the 'Workspace' option and create a new workspace.
Open a Notebook: In the Data Science section, open a new notebook to perform data analysis.
Assign a Lakehouse: Assign a Lakehouse to your notebook for storing and analyzing large datasets.
Enable Copilot: Ensure your workspace has the correct capacity to enable Copilot automatically.
Use Chat Panel: Open the Copilot chat panel by clicking the Copilot button at the top of the notebook.

Data Exploration: Generate code for plots and statistics using libraries like plotly and seaborn.
Predictive Analytics: Suggests appropriate models and tailors them to your dataset.
Learning Tool: Provides code references and advanced techniques for professional development.
Collaboration: Share saved interactions with team members to enhance productivity.
Seamless Assistance: Offers context-aware help within notebook cells and chat panel.

Preview Feature: Currently in public preview and expected to be fully available by March 2024.
Region Restrictions: Only available in specific regions; requires enabling certain settings for cross-geo data processing.
Trial SKUs: Not supported on trial SKUs; only paid SKUs (F64 or higher, or P1 or higher) are supported.
Session Termination: Requires re-running the initialization cell if the Spark session terminates.
Potential Inaccuracies: Code generation with fast-moving or recently released libraries may include inaccuracies.

Load Data as Dataframe: Maximizes Copilot's effectiveness by allowing it to understand the data's structure and content.
Use Starter Prompts: Utilize helpful starter prompts provided by Copilot to kickstart your analysis.
Clear Conversations: Use the broom icon to clear the chat panel if the current content is distracting.
Configure Privacy Settings: Use chat magics library to configure privacy settings and control data sharing.
Expand Chat Panel: Drag the sidebar to expand the chat panel for better readability of code and outputs.