How to Deploy an ML Model on Google Cloud Platform (GCP): The Complete 2026 Guide

Before You Read: The One-Minute Decision Table

Most tutorials make you read 2,000 words before answering the question you actually arrived with: which GCP deployment method should I use? Here it is upfront.

Your situation	Best path	Why
Custom model, managed infra, standard traffic	Vertex AI (Gemini Enterprise Agent Platform)	Handles auto-scaling, monitoring, and A/B traffic splits with no cluster management required
Containerised model, bursty or low traffic, cost-sensitive	Cloud Run	Scales to zero, cheapest option for unpredictable workloads
Three or more model variants, GPU inference, multi-region SLA	Google Kubernetes Engine (GKE)	Full control, Spot VMs, custom autoscaling policies
Single-function lightweight inference, event-driven	Cloud Functions Gen2	Serverless, zero ops, works well for thin wrappers or low-frequency prediction tasks

The rest of this guide covers step-by-step execution for all four paths, a 2026 cost comparison, and production best practices Ailoitte uses across client deployments.

Why GCP for ML Deployment in 2026

Google Cloud is the only major hyperscaler that builds and operates the foundation models it sells as infrastructure. That tight integration (Gemini in the stack, BigQuery for feature pipelines, TPUs for training) makes GCP a strong default for teams whose data already lives in Google Cloud. The headline numbers that matter for deployment decisions are shown below.

Data point	Figure	Source
GCP free trial credit	$300 for 90 days	Google Cloud, 2026
Vertex AI / Gemini Enterprise Agent Platform model count	200+ models including Gemini, Claude, Llama, Mistral	Google Cloud Next 2026
Vertex AI 2026 rebrand	Gemini Enterprise Agent Platform (all Vertex AI APIs and infrastructure intact)	Google Cloud, April 2026
Gemini 2.5 Flash-Lite token pricing	$0.10 per 1M input tokens, $0.40 per 1M output tokens	Google Cloud Pricing, June 2026
Cloud Run free tier	2M requests/month, 360,000 GiB-seconds/month, 180,000 vCPU-seconds/month	Google Cloud Run Pricing
Vertex AI Generative AI SDK deprecation	vertexai.generative_models deprecated June 24, 2025; removed June 24, 2026. Core deployment SDK (google-cloud-aiplatform) is unaffected.	Google Cloud Deprecations

Important 2026 update: at Google Cloud Next 2026 in Las Vegas, Google rebranded Vertex AI as the Gemini Enterprise Agent Platform. The underlying Vertex AI APIs, Model Registry, and deployment infrastructure are unchanged; the rebrand consolidates Vertex AI and Agentspace into one product. All code in this guide uses aiplatform.googleapis.com, which remains valid. New Gemini-specific features are being shipped exclusively through the Gemini Enterprise Agent Platform going forward. Source: Google Cloud official product page.

Prerequisites

Complete these steps before running any deployment command. Teams that skip IAM setup typically lose two hours to permission errors on first deployment.

1. Create a Google Cloud project and enable billing. See: cloud.google.com/resource-manager
2. Enable required APIs:

gcloud services enable aiplatform.googleapis.com \                        storage.googleapis.com \                        run.googleapis.com \                        container.googleapis.com \                        cloudfunctions.googleapis.com

nstall and initialise the gcloud CLI. Install guide: cloud.google.com/sdk/docs/install

gcloud auth login gcloud config set project YOUR_PROJECT_ID gcloud config set ai/region us-central1

Create a service account for your deployment workload:

gcloud iam service-accounts create ml-deploy-sa \   –display-name=’ML Deployment Service Account’ gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \   –member=’serviceAccount:ml-deploy-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com’ \   –role=’roles/aiplatform.user’ gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \   –member=’serviceAccount:ml-deploy-sa@YOUR_PROJECT_ID.iam.gserviceaccount.com’ \   –role=’roles/storage.objectAdmin’

Create a Cloud Storage bucket for model artefacts:

gcloud storage buckets create gs://YOUR_PROJECT_ID-models \   –location=us-central1 \   –uniform-bucket-level-access

Preparing Your Model for Deployment

Every GCP deployment path requires the model exported in a compatible format and staged in Cloud Storage. The table below maps framework to required export format.

Framework	Export format	Export command
TensorFlow 2.x	SavedModel	model.save(‘gs://YOUR_BUCKET/model/1/’)
PyTorch	.pt or .pth (TorchServe for serving)	torch.save(model.state_dict(), ‘model.pt’)
Scikit-learn	.pkl or .joblib	joblib.dump(model, ‘model.joblib’)
XGBoost	.bst	model.save_model(‘model.bst’)
ONNX (framework-agnostic)	.onnx	torch.onnx.export(model, …) or tf2onnx

Upload your exported model to GCS:

# TensorFlow SavedModel (directory upload) gcloud storage cp -r ./saved_model/ gs://YOUR_PROJECT_ID-models/my-model/v1/ # Single file (PyTorch, sklearn, XGBoost) gcloud storage cp model.joblib gs://YOUR_PROJECT_ID-models/my-model/v1/model.joblib

Path 1: Vertex AI (Recommended for Most Production Deployments)

Vertex AI (now part of the Gemini Enterprise Agent Platform) is Google’s fully managed ML platform and the right default for production deployments. It handles auto-scaling, traffic splitting for A/B testing, model monitoring, and Explainable AI out of the box. Pricing for custom-trained models is per prediction node-hour: an n1-standard-4 node costs approximately $0.223/hour in us-central1 as of June 2026 (source: nops.io Vertex AI Pricing Guide 2026).

Step 1 – Register the model in Vertex AI Model Registry

gcloud ai models upload \   –region=us-central1 \   –display-name=’my-classifier-v1′ \   –container-image-uri=us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-13:latest \   –artifact-uri=gs://YOUR_PROJECT_ID-models/my-model/v1/ # Note the MODEL_ID in the output — required in Step 3

Step 2 – Create an endpoint

gcloud ai endpoints create \   –region=us-central1 \   –display-name=’my-classifier-endpoint’ # Note the ENDPOINT_ID in the output

Step 3 – Deploy the model to the endpoint

gcloud ai endpoints deploy-model ENDPOINT_ID \   –region=us-central1 \   –model=MODEL_ID \   –display-name=’my-classifier-v1′ \   –machine-type=n1-standard-4 \   –min-replica-count=1 \   –max-replica-count=5 \   –traffic-split=0=100 # Full flag reference: https://cloud.google.com/sdk/gcloud/reference/ai/endpoints/deploy-model

Step 4 – Test with a prediction request

gcloud ai endpoints predict ENDPOINT_ID \   –region=us-central1 \   –json-request='{“instances”: [[5.1, 3.5, 1.4, 0.2]]}’

For Python SDK deployment (preferred in CI/CD pipelines), see the official Vertex AI custom model deployment sample at cloud.google.com/vertex-ai/docs/samples.

A/B Traffic Splitting (Vertex AI only)

Vertex AI supports multi-model traffic splitting on a single endpoint, routing a percentage of traffic to each deployed model version for safe rollouts. Cloud Run does not support this without a custom load-balancer layer.

# After deploying a second model version, update traffic split: gcloud ai endpoints deploy-model ENDPOINT_ID \   –region=us-central1 \   –model=MODEL_ID_V2 \   –display-name=’my-classifier-v2′ \   –machine-type=n1-standard-4 \   –min-replica-count=1 \   –traffic-split=DEPLOYED_MODEL_ID_V1=80,0=20 # Routes 80% of traffic to v1, 20% to the newly deployed v2

Path 2: Cloud Run (Best for Cost-Sensitive or Variable Workloads)

Cloud Run deploys containerised applications in a fully managed serverless environment. It scales to zero when idle, meaning you pay nothing for a model that receives no traffic. For development, staging, and low-to-medium production workloads, Cloud Run is almost always cheaper than a permanently running Vertex AI endpoint. Maximum instance size is 32 GiB memory and 8 vCPUs (source: Cloud Run memory docs, updated June 2026; Cloud Run CPU docs).

Step 1 – Wrap your model in a FastAPI server

# app.py from fastapi import FastAPI import joblib, numpy as np from pydantic import BaseModel app = FastAPI() model = joblib.load(‘model.joblib’) class PredictRequest(BaseModel):     features: list[float] @app.post(‘/predict’) def predict(req: PredictRequest):     arr = np.array(req.features).reshape(1, -1)     prediction = model.predict(arr).tolist()     return {‘prediction’: prediction} @app.get(‘/health’) def health():     return {‘status’: ‘ok’}

Step 2 – Containerise with Docker

# Dockerfile FROM python:3.12-slim WORKDIR /app COPY requirements.txt . RUN pip install –no-cache-dir -r requirements.txt COPY model.joblib . COPY app.py . EXPOSE 8080 CMD [“uvicorn”, “app:app”, “–host”, “0.0.0.0”, “–port”, “8080”] # requirements.txt fastapi==0.111.0 uvicorn==0.30.1 scikit-learn==1.5.0 numpy==1.26.4 joblib==1.4.2

Step 3 – Build, push, and deploy

# Build and push to Artifact Registry gcloud builds submit –tag us-central1-docker.pkg.dev/YOUR_PROJECT_ID/ml-repo/my-model:v1 # Deploy to Cloud Run gcloud run deploy my-model-service \   –image=us-central1-docker.pkg.dev/YOUR_PROJECT_ID/ml-repo/my-model:v1 \   –region=us-central1 \   –platform=managed \   –allow-unauthenticated \   –memory=2Gi \   –cpu=2 \   –min-instances=0 \   –max-instances=10 \   –concurrency=80

Step 4 – Test the endpoint

SERVICE_URL=$(gcloud run services describe my-model-service \   –region=us-central1 –format=’value(status.url)’) curl -X POST $SERVICE_URL/predict \   -H ‘Content-Type: application/json’ \   -d ‘{“features”: [5.1, 3.5, 1.4, 0.2]}’

Path 3: Google Kubernetes Engine (Best for Scale, GPU Inference, Multi-Model Systems)

GKE is the right choice when you need GPU-accelerated inference, multi-model serving on shared infrastructure, or fine-grained autoscaling control. It has higher operational overhead than Vertex AI or Cloud Run; plan for a dedicated MLOps engineer or use Ailoitte’s AI/ML development services if your team does not have Kubernetes expertise.

If you are deploying AI agents that call Vertex AI as a tool, see the GKE + Vertex AI ADK guide at cloud.google.com/kubernetes-engine/docs/tutorials/agentic-adk-vertex.

Step 1 – Create a GKE Autopilot cluster

gcloud container clusters create-auto ml-cluster \   –location=us-central1 \   –project=YOUR_PROJECT_ID gcloud container clusters get-credentials ml-cluster \   –location=us-central1

Step 2 – Create a Kubernetes Deployment manifest

# deployment.yaml apiVersion: apps/v1 kind: Deployment metadata:   name: ml-model-serving   labels:     app: ml-model spec:   replicas: 2   selector:     matchLabels:       app: ml-model   template:     metadata:       labels:         app: ml-model     spec:       containers:       – name: model-server         image: us-central1-docker.pkg.dev/YOUR_PROJECT_ID/ml-repo/my-model:v1         ports:         – containerPort: 8080         resources:           requests:             memory: ‘2Gi’             cpu: ‘1000m’           limits:             memory: ‘4Gi’             cpu: ‘2000m’         readinessProbe:           httpGet:             path: /health             port: 8080           initialDelaySeconds: 10           periodSeconds: 5

Step 3 – Expose as a Service and apply

# service.yaml apiVersion: v1 kind: Service metadata:   name: ml-model-service spec:   selector:     app: ml-model   ports:   – port: 80     targetPort: 8080   type: LoadBalancer kubectl apply -f deployment.yaml kubectl apply -f service.yaml kubectl rollout status deployment/ml-model-serving

Step 4 – Add Horizontal Pod Autoscaler

kubectl autoscale deployment ml-model-serving \   –cpu-percent=70 \   –min=2 \   –max=20

For GPU-accelerated inference (large language models, computer vision), see the GKE GPU node pool documentation: cloud.google.com/kubernetes-engine/docs/how-to/gpus.

Path 4: Cloud Functions Gen2 (Best for Lightweight, Event-Driven Inference)

Cloud Functions Gen2 is the simplest deployment option: no containers to manage, no cluster to operate. It works well for models under 500 MB, low-frequency inference, and thin wrappers around pre-built API endpoints. Gen2 instances support up to 16 GiB RAM and 4 vCPUs in GA, with higher configurations (32 GiB / 8 vCPU) available for workloads requiring more memory. Source: Google Cloud Blog, Cloud Functions 2nd gen GA.

Step 1 – Write the function

# main.py import functions_framework import joblib, numpy as np, json from google.cloud import storage # Load model once at cold start; cached across warm invocations client = storage.Client() bucket = client.bucket(‘YOUR_PROJECT_ID-models’) blob = bucket.blob(‘my-model/v1/model.joblib’) blob.download_to_filename(‘/tmp/model.joblib’) model = joblib.load(‘/tmp/model.joblib’) @functions_framework.http def predict(request):     data = request.get_json()     features = np.array(data[‘features’]).reshape(1, -1)     result = model.predict(features).tolist()     return json.dumps({‘prediction’: result})

Step 2 – Deploy

gcloud functions deploy ml-predict \   –gen2 \   –runtime=python312 \   –region=us-central1 \   –source=. \   –entry-point=predict \   –trigger-http \   –memory=1Gi \   –timeout=60s \   –allow-unauthenticated

2026 Cost Comparison: Which Path Is Cheapest for Your Workload

Estimates below assume us-central1 region and 10,000 prediction requests per day. Vertex AI node-hour pricing sourced from nops.io; Cloud Run pricing from cloud.google.com/run/pricing; Gemini token pricing from cloud.google.com/vertex-ai/pricing.

Deployment path	Idle cost / month	~10K req/day cost / month	Best for
Vertex AI endpoint (n1-standard-4, 1 replica)	~$161	~$161 plus prediction node hours	Managed inference, A/B testing, MLOps pipeline
Cloud Run (scale to zero, 2 vCPU / 2 GiB)	$0	~$15-40	Variable or low traffic, cost-optimisation
Cloud Run (1 min-instance warm)	~$20-40	~$40-70	Low-latency APIs with burst traffic patterns
GKE Autopilot (2 replicas)	~$80-120	~$80-150	GPU inference, multi-model, custom autoscaling
Cloud Functions Gen2 (1 GiB, 1 vCPU)	$0	~$5-15	Lightweight models, event-triggered workloads

For teams deploying Gemini or other foundation models via the Vertex AI API (not custom models), pricing is token-based. Gemini 2.5 Flash-Lite costs $0.10 per 1M input tokens and $0.40 per 1M output tokens as of June 2026. Vertex AI does not charge a separate endpoint fee for hosted foundation models. Source: CloudZero Vertex AI Pricing Guide 2026.

Testing Your Deployed Model

Online prediction testing

After deployment, always test with known inputs before routing production traffic.

# Vertex AI gcloud ai endpoints predict ENDPOINT_ID \   –region=us-central1 \   –json-request=test_payload.json # Cloud Run or GKE curl -X POST https://YOUR-SERVICE-URL/predict \   -H ‘Content-Type: application/json’ \   -d @test_payload.json

Load testing

Use Locust or k6 to verify your endpoint handles peak load. Test at 2x expected peak before enabling in production. Monitor Cloud Monitoring for latency percentiles and error rate. A p99 latency target of under 500ms is a common baseline for real-time ML inference APIs.

Production MLOps Best Practices

1. Model monitoring and data drift detection

Vertex AI includes built-in model monitoring for data drift and prediction skew. Enable it on your endpoint to detect when incoming feature distributions shift away from training data. Full setup guide: cloud.google.com/vertex-ai/docs/model-monitoring.

# Enable monitoring via gcloud (recommended approach for 2026) gcloud ai model-monitors create \   –region=us-central1 \   –display-name=’my-classifier-monitor’ \   –model=projects/YOUR_PROJECT_ID/locations/us-central1/models/MODEL_ID # Then configure notification channels and thresholds in the Cloud Console # or via the REST API: https://cloud.google.com/vertex-ai/docs/model-monitoring

2. Version control with Model Registry

Tag every model upload with a semantic version and git commit SHA. Vertex AI Model Registry supports model versioning to control traffic routing without hardcoding IDs.

gcloud ai models upload \   –region=us-central1 \   –display-name=’my-classifier-v2.1.0′ \   –description=’Retrained on Q2 2026 data, +3.2% accuracy on holdout set’ \   –artifact-uri=gs://YOUR_PROJECT_ID-models/my-model/v2.1.0/ \   –container-image-uri=us-docker.pkg.dev/vertex-ai/prediction/sklearn-cpu.1-5:latest \   –labels=’version=2-1-0,git_sha=a3f92b1,env=production’

3. Security hardening

• Use Workload Identity for GKE pods to access GCP services without JSON key files. See: cloud.google.com/kubernetes-engine/docs/how-to/workload-identity
• Enable VPC Service Controls on Vertex AI endpoints for data-sensitive workloads (HIPAA, PCI-DSS). See: cloud.google.com/vpc-service-controls
• Use Customer-Managed Encryption Keys (CMEK) for model artefacts in Cloud Storage. See: cloud.google.com/kms
• Remove –allow-unauthenticated from Cloud Run for internal services. Use Cloud IAP instead: cloud.google.com/iap
• Enable Cloud Audit Logs for aiplatform.googleapis.com to track all prediction requests and model operations.

4. Cost controls

• Set a budget alert in Google Cloud Billing. Vertex AI has no hard spending cap, so billing alerts are the only guardrail.
• Auto-undeploy idle Vertex AI endpoints using Cloud Scheduler and Cloud Functions. Check metrics every 6 hours; undeploy any endpoint with zero requests tagged as dev or staging.
• Idle n1-standard-4 endpoints cost ~$161/month each. Removing 15 idle dev endpoints saves approximately $1,606/month (source: nops.io).
• Use Spot VMs in GKE for batch inference jobs to reduce compute costs significantly versus on-demand pricing. See: cloud.google.com/kubernetes-engine/docs/concepts/spot-vms

Common Deployment Challenges and Fixes

Challenge	Symptom	Fix
Framework version mismatch	Model returns unexpected outputs or 500 errors on edge inputs	Pin all dependency versions in requirements.txt; match training and serving environments exactly
Cold start latency on Cloud Run or Functions	First request after idle period takes 8-15 seconds	Set min-instances=1 for latency-sensitive endpoints; pre-load the model outside the request handler
IAM permission errors	403 on prediction requests or GCS access denied	Verify service account has roles/aiplatform.user and roles/storage.objectViewer on the bucket
Data drift degrading accuracy	Model accuracy degrades weeks after deployment	Enable Vertex AI Model Monitoring; schedule weekly retraining triggers via Vertex AI Pipelines
Cost overrun on idle endpoints	Unexpectedly high GCP bill	Implement auto-undeployment for dev/staging; use Cloud Run instead of Vertex AI for non-production
Container startup failures in GKE	CrashLoopBackOff on pod startup	Run kubectl logs POD_NAME; verify ENTRYPOINT in Dockerfile; test the container image locally first

How Ailoitte Approaches GCP ML Deployments

Ailoitte’s AI/ML development practice and machine learning development services have deployed custom models on GCP across regulated industries including healthcare, fintech, and edtech. The default deployment sequence for new client projects follows this path:

1. Cloud Run for the first production deployment: low ops overhead, scales to zero, cheapest path to a live REST endpoint.
2. Vertex AI when the model needs A/B testing, automated model monitoring, or an SLA-backed inference guarantee.
3. GKE when the client runs three or more model variants, needs GPU inference, or requires multi-region availability.

For clients building generative AI applications and AI agents, we integrate GCP deployments with Vertex AI Agent Builder (now part of Gemini Enterprise Agent Platform), enabling the deployed model to serve as a tool-calling endpoint within a multi-agent workflow.

Our AI consulting services include a GCP deployment audit covering IAM hygiene, endpoint idle-cost analysis, model monitoring setup, and CI/CD pipeline review. Most clients reduce their first-year GCP inference costs by 35-50% after an audit.

If you need Python developers experienced in MLOps on GCP (Vertex AI Pipelines, FastAPI serving on Cloud Run, GKE deployment automation), Ailoitte engineering pods can embed directly in your team.

Conclusion

Deploying an ML model on Google Cloud Platform in 2026 is faster and cheaper than it was two years ago, but only if you choose the right path for your workload from the start.

Decision context	Recommended path
Teams new to GCP ML deployment, standard traffic patterns	Vertex AI
Variable or bursty traffic, cost-constrained budgets	Cloud Run with scale-to-zero
GPU inference, multi-model systems, enterprise SLA	GKE Autopilot
Lightweight inference, event-driven backend	Cloud Functions Gen2

The single most expensive mistake in GCP ML deployment is not the wrong framework or machine type; it is leaving dev and staging endpoints running 24/7 on Vertex AI. Implement idle-endpoint cleanup on day one and your GCP bill will stay predictable as your model footprint grows.

Ailoitte delivers end-to-end AI/ML development, machine learning model development, and AI transformation services including model deployment, MLOps pipelines, and GCP architecture reviews. To get a second opinion on your deployment architecture before going live, reach out to our team.

Hide Comments