gke-ai-troubleshooting-tpu-dynamic-slices-monitoring
Monitors, troubleshoots, and manages GKE TPU Dynamic Slices custom resources
Install / Use
npx skills add google/skills --skill gke-ai-troubleshooting-tpu-dynamic-slices-monitoringInstalls into whichever agent you are using.
SKILL.md
Installable skill definition
Quality Score
Category
OperationsSupported Platforms
Tags
Our assessment of gke-ai-troubleshooting-tpu-dynamic-slices-monitoring
gke-ai-troubleshooting-tpu-dynamic-slices-monitoring scores 87/100 on our quality scale, 430th of 746 Operations skills we index.
Its SKILL.md is 7.6 KB long, well organised into 13 sections and no code examples: a thorough specification that gives an agent plenty to work with.
With 20,340 GitHub stars, it is one of the more widely adopted skills in the catalogue.
Maintenance, license and trust
- The repository was last updated about 2 months ago, so gke-ai-troubleshooting-tpu-dynamic-slices-monitoring is actively maintained.
- Our last check on 2026-11-02 found the source still online.
- It is released under the Apache-2.0 license, a permissive license that allows use, modification and commercial use with attribution.
- Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.
Safety scan
No issues foundOur scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.
AI review by kimi-k2.7-code on 2026-09-27. Automated pattern scan on 2026-09-26. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.
gke-ai-troubleshooting-tpu-dynamic-slices-monitoring compared with similar skills
All 4 of these similar skills score higher than gke-ai-troubleshooting-tpu-dynamic-slices-monitoring; compare them before choosing.
| Skill | Score | Stars | Updated | Format |
|---|---|---|---|---|
| gke-ai-troubleshooting-tpu-dynamic-slices-monitoring (this skill)by google | 87 | 20.3k | 54d ago | SKILL.md |
| x402by browser-use | 100 | 121.0k | 3d ago | SKILL.md |
| algorithmic-artby anthropics | 100 | 177.9k | 55d ago | SKILL.md |
| pptxby anthropics | 100 | 177.9k | 55d ago | SKILL.md |
| designby nextlevelbuilder | 100 | 149.0k | 6d ago | SKILL.md |
Frequently asked questions
- How do I install gke-ai-troubleshooting-tpu-dynamic-slices-monitoring?
- Run
npx skills add google/skills --skill gke-ai-troubleshooting-tpu-dynamic-slices-monitoring. The install tabs above show the steps for each supported agent. - Which AI agents does gke-ai-troubleshooting-tpu-dynamic-slices-monitoring work with?
- It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
- Is gke-ai-troubleshooting-tpu-dynamic-slices-monitoring safe to use?
- Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is Apache-2.0-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
- Is gke-ai-troubleshooting-tpu-dynamic-slices-monitoring still maintained?
- The repository was last updated about 2 months ago, so gke-ai-troubleshooting-tpu-dynamic-slices-monitoring is actively maintained.
Skill content
View source on GitHubname: gke-ai-troubleshooting-tpu-dynamic-slices-monitoring description: >- Monitors, troubleshoots, and manages GKE TPU Dynamic Slices custom resources. Use when checking TPU slice lifecycle states, troubleshooting slice provisioning failures, validating single-slice or multi-slice (JobSet) workload manifests, or safely patching stuck finalizers and disabling the slice controller. Don't use for generic GKE cluster node pool creation or standard non-TPU workload management (use gke-basics or gke-cluster-creation instead). metadata: version: "1.0.0" category: Containers
GKE TPU Dynamic Slices Monitoring & Management
Monitors the status of TPU Slice custom resources, troubleshoots provisioning failures, validates workload manifests on dynamic slices, and performs cleanups.
Prerequisites
- Cloud Logging enabled for the project.
kubectlandgcloudCLIs configured to access the GKE cluster.
Diagnostic Workflow
Step 0: Context Acquisition & Time Window Definition
Gather project, cluster, and slice context using cluster tools or the following parameters:
- Project ID:
{project_id}(e.g.,my-gcp-project) - Cluster Name:
{cluster_name}(e.g.,tpu-cluster) - Region/Zone:
{location}(e.g.,us-central1-a) - Slice Name:
{slice_name}(e.g.,test-slice) - Issue Time:
{timestamp}(Optional; default to the last 30 minutes window[T - 30m]to[T + 30m])
Step 1: Describe the Slice Custom Resource [Low Risk]
When asked to inspect, troubleshoot, or check a slice status, immediately execute kubectl describe slice {slice_name} using available cluster tools to perform the inspection. Parse the resulting Status.Conditions output against the condition table below to diagnose the exact state and provide concrete recommendations.
-
Command:
kubectl describe slice {slice_name}
State & Reason Analysis
Analyze the Status.Conditions (especially Type: Ready and its Reason and
Status):
| Lifecycle State / Reason | Meaning | Recommended Action |
| :--- | :--- | :--- |
| SliceNotCreated | GKE Slice Controller is initializing the slice and performing resource checks. | Wait a few minutes and re-check slice status. |
| SliceCreationFailed | Prerequisites validation failed (e.g., selected nodes don't exist, nodes are already used by another slice, or the topology doesn't match the number of partitions). | Verify selected nodes exist, are unallocated, and topology matches partition count. |
| ACTIVATING | GKE is actively forming and provisioning the TPU slice. | Monitor node provisioning. |
| ACTIVE | The TPU slice is successfully formed and ready to host workloads. | Proceed to deploy or check workloads. |
| ACTIVE_DEGRADED | The slice is usable, but one or more sub-blocks are degraded. | Monitor workload logs for interconnect or device errors. Check faulty node VMs. |
| FAILED | GKE failed to form the TPU slice (e.g., selected nodes are not part of the same reservation block). | Ensure all selected nodes belong to the same reservation block. |
| DEACTIVATING | The slice is dismantling (triggered by user deletion or a critical systemic failure). | Wait for dismantling to finish, or patch finalizers if stuck. |
| INCOMPLETE | The terminal phase before the Slice CR is deleted from the cluster. | No action required; the resource will be removed shortly. |
Provisioning Failure Troubleshooting Checklist
When investigating slice creation or provisioning failures (SliceCreationFailed or FAILED), perform the following verification steps:
- Node Existence & Allocation Check: Verify that the selected TPU nodes exist in the cluster and are not already allocated to another slice (
kubectl get nodes -l cloud.google.com/gke-tpu-slice,kubectl get slice -A). - Topology Alignment: Confirm that the partition count matches the requested topology dimensions (e.g. topology
2x2requires 4 nodes). - Reservation Block Alignment Check: Confirm that all selected TPU nodes belong to the same reservation and reservation block.
Step 2: Verify Workload Specification [Low Risk]
Ensure workload manifests are configured correctly to target the dynamic slice.
1. Single-Slice Workload Requirements
Check that the Pod template contains the following annotations and selectors:
- Annotations:
cloud.google.com/gke-tpu-slice-topology: "{topology}"(e.g.,"4x4x4")
- NodeSelector:
cloud.google.com/gke-tpu-topology: "{topology}"(e.g.,"4x4x4")cloud.google.com/gke-tpu-accelerator: "{accelerator_type}"(e.g.,"tpu7x")cloud.google.com/gke-tpu-slice: "{slice_name}"(e.g.,"test-slice")
2. Multi-Slice (JobSet) Workload Requirements
If deploying a multi-slice JobSet, verify:
- JobSet Annotation:
alpha.jobset.sigs.k8s.io/exclusive-topology: cloud.google.com/gke-tpu-slice
- Pod Template Annotations:
cloud.google.com/gke-tpu-slice-topology: "{topology}"
- Pod Template NodeSelector:
cloud.google.com/gke-tpu-topology: "{topology}"cloud.google.com/gke-tpu-accelerator: "{accelerator_type}"- Note: Do NOT manually specify
cloud.google.com/gke-tpu-slicein the nodeSelector; JobSet handles slice assignment automatically.
Resolution & Management Workflow
Resolution 1: Force Delete a Stuck Slice [High Risk]
If a slice is stuck in DEACTIVATING or deletion hangs indefinitely due to stuck finalizers:
-
Identify Cause: Explain that finalizers on the slice resource (
metadata.finalizers) are preventing Kubernetes from completing resource deletion. -
Propose Resolution: Propose removing finalizers from the metadata path (
/metadata/finalizers) using a JSON patch operation:kubectl patch slice {slice_name} --type json -p='[{"op": "remove", "path": "/metadata/finalizers"}]' -
Provide Warning: Explicitly warn the user that removing finalizers bypasses standard controller dismantling and may leave underlying VM, network, or accelerator resources uncleaned or orphaned.
-
CRITICAL SAFETY MANDATE: The response MUST explicitly ask the user for confirmation (e.g. "Removing finalizers on
/metadata/finalizersvia JSON patch is a high-risk operation that may leave orphaned resources. Do you confirm you want to apply this patch to slice{slice_name}?") and pause for user confirmation before applying or executing the patch.
Resolution 2: Disable and Clean Up Slice Controller [High Risk]
If dynamic slicing needs to be disabled:
-
Check for existing Slices:
kubectl get slice -AEnsure all slices are deleted before disabling the controller.
-
Disable Slice Controller via gcloud:
gcloud container clusters update {cluster_name} \ --location={location} \ --no-enable-slice-controller -
Delete the Slice CRD:
kubectl delete crd slices.accelerator.gke.io -
Clean up Node Labels: Remove GKE TPU Slice labels from all nodes in the cluster:
kubectl label nodes --all cloud.google.com/gke-tpu-slice- cloud.google.com/gke-tpu-slice-topology-
- Safety Rule: Propose the exact commands and confirm before executing disabling or destructive cleanup steps.
Related Skills
x402
121.0kSet up Browser Use Cloud payments with x402 — pay per request from a crypto wallet (USDC on Base mainnet), no signup or API key.
algorithmic-art
177.9kCreating algorithmic art using p5.js with seeded randomness and interactive parameter exploration. Use this when users request creating art using code, generative art, algorithmic art, flow fields, or particle systems.
pptx
177.9kUse this skill any time a .pptx or .potx file is involved in any way — as input, output, or both. This includes: creating slide decks, pitch decks, or presentations; reading, parsing, or extracting text from any .pptx or .potx file (even if the extracted content will be used elsewhere, like in an em…
design
149.0kComprehensive design skill: brand identity, design tokens, UI styling, logo generation (55 styles, Gemini, Atlas Cloud, or MuAPI AI), corporate identity program (50 deliverables, CIP mockups), HTML presentations (Chart.js), banner design (22 styles, social/ads/web/print), icon design (15 styles, SVG…
Languages
Trust signals
From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.
