SOP-003: Model Creation

DOCUMENT CONTROL

Field	Value
SOP ID	SOP-003
Version	1.0
Status	Active

INFO

Document Control

Document	Model Creation
Version	1.0
Date	2023-04-20
Owner	AI Engineering Team

Purpose

This Standard Operating Procedure (SOP) provides a detailed guide on how to create and train a machine learning model using Vertex AI. The procedure outlined in this document will help ensure consistency and quality in the model creation process.

Procedure Flowchart

┌──────────────────────┐
│ 1. Prepare Dataset   │
└──────────────────────┘
       ▼
┌──────────────────────┐
│ 2. Define Model      │
└──────────────────────┘
       ▼
┌──────────────────────┐
│ 3. Train Model       │
└──────────────────────┘
       ▼
┌──────────────────────┐
│ 4. Evaluate Model    │
└──────────────────────┘
       ▼
┌──────────────────────┐
│ 5. Deploy Model      │
└──────────────────────┘

Step-by-Step Procedure

1. Prepare Dataset

WARNING

Ensure that the dataset is clean, formatted correctly, and split into training, validation, and test sets.

1. Load the dataset into a Pandas DataFrame or a TensorFlow Dataset object.
2. Preprocess the data, including handling missing values, encoding categorical variables, and scaling numerical features.
3. Split the dataset into training, validation, and test sets.

Example code:

import pandas as pd
from sklearn.model_selection import train_test_split

# Load the dataset
data = pd.read_csv('dataset.csv')

# Preprocess the data
data = preprocess_data(data)

# Split the dataset
X_train, X_test, y_train, y_test = train_test_split(data.drop('target', axis=1), data['target'], test_size=0.2, random_state=42)

2. Define Model

INFO

Choose an appropriate machine learning model based on the problem you're trying to solve (e.g., classification, regression, or clustering).

1. Import the necessary libraries and modules from Vertex AI.
2. Define the model architecture using the appropriate Vertex AI API (e.g., TabularDatasetDataSource, TabularPredictionJob, AutoMLTabularTrainingJob).
3. Configure the model hyperparameters and training settings.

Example code:

from google.cloud import aiplatform

# Define the model
model = aiplatform.AutoMLTabularTrainingJob(
    display_name='my-model',
    optimization_prediction_type='classification',
    column_transformations=[
        aiplatform.feature_engineering.numeric_column('feature1'),
        aiplatform.feature_engineering.categorical_column('feature2')
    ],
    training_fraction_split=0.8,
    validation_fraction_split=0.1,
    test_fraction_split=0.1
)

# Configure the training settings
model.run(
    dataset=my_dataset,
    model_display_name='my-model',
    training_args={
        'budget_milli_node_hours': 1000,
        'max_trial_count': 5
    }
)

3. Train Model

WARNING

Ensure that you have the necessary permissions and resources (e.g., compute instances, storage) to train the model on Vertex AI.

1. Start the model training process on Vertex AI.
2. Monitor the training progress and logs.
3. Wait for the training to complete.

Example code:

# Start the training
model_training_job = model.run()

# Monitor the training progress
model_training_job.wait()
print(model_training_job.state)

4. Evaluate Model

INFO

Evaluate the model's performance using appropriate metrics (e.g., accuracy, precision, recall, F1-score) on the test dataset.

1. Use the trained model to make predictions on the test dataset.
2. Calculate the relevant performance metrics.
3. Analyze the model's performance and determine if it meets the required criteria.

Example code:

# Make predictions on the test dataset
y_pred = model.predict(X_test)

# Evaluate the model's performance
from sklearn.metrics import accuracy_score, precision_score, recall_score, f1_score
print(f'Accuracy: {accuracy_score(y_test, y_pred)}')
print(f'Precision: {precision_score(y_test, y_pred)}')
print(f'Recall: {recall_score(y_test, y_pred)}')
print(f'F1-score: {f1_score(y_test, y_pred)}')

5. Deploy Model

DANGER

Ensure that you have the necessary permissions and resources to deploy the model on Vertex AI.

1. Prepare the model for deployment (e.g., save the model, export it to a format compatible with Vertex AI).
2. Deploy the model to a Vertex AI endpoint.
3. Test the deployed model by making predictions.

Example code:

# Deploy the model
endpoint = model.deploy(
    machine_type='n1-standard-4',
    min_replica_count=1,
    max_replica_count=5
)

# Test the deployed model
response = endpoint.predict(instances=X_test.iloc[:5].to_dict('records'))
print(response.predictions)

Verification Checklist

• [ ] Dataset is prepared and split into training, validation, and test sets.
• [ ] Model architecture is defined using the appropriate Vertex AI API.
• [ ] Model hyperparameters and training settings are configured.
• [ ] Model is trained on Vertex AI.
• [ ] Model is evaluated on the test dataset, and the performance metrics meet the required criteria.
• [ ] Model is deployed to a Vertex AI endpoint.
• [ ] Deployed model is tested, and the predictions are verified.

Troubleshooting

Issue	Possible Cause	Solution
Dataset not loading correctly	Incorrect file path or format	Check the file path and ensure the dataset is in a supported format (e.g., CSV, Parquet)
Model training fails	Insufficient resources or permissions	Check the available resources (e.g., compute instances, storage) and ensure you have the necessary permissions to use Vertex AI
Model performance is poor	Inappropriate model selection or hyperparameters	Experiment with different model architectures or hyperparameters, and evaluate the model's performance on the validation set
Deployment fails	Incorrect model format or deployment settings	Ensure the model is exported in a format compatible with Vertex AI, and the deployment settings (e.g., machine type, replica count) are correct

See Also

• SOP-001: Initial Setup
• SOP-002: Configuration
• SOP-004: Model Deployment