kubectl debug: support debugging nodes #92310

verb · 2020-06-19T15:55:37Z

What type of PR is this?
/kind feature
/sig cli
/priority important-longterm

What this PR does / why we need it:

Which issue(s) this PR fixes:
Fixes #89306

Special notes for your reviewer: This PR includes commits from #90094 and should not merge prior to it. To see the changes introduced by this change alone, use the individual commits view.

Does this PR introduce a user-facing change?:

`kubectl alpha debug` now supports debugging nodes by creating a debugging container running in the node's host namespaces.

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

- [KEP]: https://git.k8s.io/enhancements/keps/sig-cli/20190805-kubectl-debug.md#node-troubleshooting-with-privileged-containers

tedyu · 2020-06-19T20:13:21Z

staging/src/k8s.io/kubectl/pkg/cmd/debug/debug.go

+		names[ref.Name] = ref
+	}
+	for i := range pod.Spec.InitContainers {
+		ref := &pod.Spec.Containers[i]


Is there a typo here ?
In worst case, there may be index out of bounds exception.

Hi @tedyu, this PR is based on #90094, and I think your comment is for the underlying commit. Could you follow up in #90094?

soltysh

A few nits

soltysh · 2020-06-22T15:48:50Z

staging/src/k8s.io/kubectl/pkg/cmd/debug/debug.go

+		The action taken by 'debug' varies depending on what resource is specified. Supported
+		actions include:
+
+		* Pod: Create a copy of an existing pod with certain attributes changed, for example


I'd call it workload, since it's not a problem to get a pod from a deployment/daemon set/job, etc.

Make sense, and it might be nice to support automatically resolving simple workloads to pods. Updated.

soltysh · 2020-06-22T15:49:31Z

staging/src/k8s.io/kubectl/pkg/cmd/debug/debug.go

@@ -221,6 +241,11 @@ func (o *DebugOptions) Validate(cmd *cobra.Command) error {
 		return fmt.Errorf("invalid image pull policy: %s", o.PullPolicy)
 	}

+	// Target
+	if len(o.Target) > 0 && len(o.CopyTo) > 0 {
+		return fmt.Errorf("--target incompatible with --copy-to. Use --share-processes instead.")


is incompatible

soltysh · 2020-06-22T15:50:16Z

staging/src/k8s.io/kubectl/pkg/cmd/debug/debug.go

+	if err != nil {
+		return nil, "", err
+	}
+	klog.V(2).Infof("node debugPod create returned pod: %#v", newPod)


Debug message, I'd remove.

soltysh · 2020-06-22T15:52:08Z

staging/src/k8s.io/kubectl/pkg/cmd/debug/debug.go

+					TTY:                      o.TTY,
+					VolumeMounts: []corev1.VolumeMount{
+						{
+							MountPath: "/node",


Nit: aybe just /host here, and host below

SGTM. I've updated this and the other messages to use "node" only in the context of the API object and "host" in the context of namespaces and filesystem. Now that you point it out, I realize this is more in line with how these terms are used in the rest of the API.

When called with a node target, `kubectl debug` will create a run-once pod in the target node's namespaces.

verb · 2020-07-09T10:52:49Z

/retest

soltysh

/lgtm
/approve
/milestone v1.19

k8s-ci-robot · 2020-07-09T17:54:55Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: soltysh, verb

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~staging/src/k8s.io/kubectl/OWNERS~~ [soltysh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

fejta-bot · 2020-07-10T16:11:30Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

verb · 2020-07-10T19:00:17Z

/retest

fejta-bot · 2020-07-10T22:29:39Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-11T01:38:12Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-11T10:02:21Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

verb · 2020-07-11T13:38:14Z

/retest

fejta-bot · 2020-07-11T18:26:13Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-12T01:05:15Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

joelsmith · 2020-07-12T02:21:08Z

/retest

joelsmith · 2020-07-12T03:13:19Z

/retest

fejta-bot · 2020-07-12T06:41:10Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-12T14:23:09Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2020-07-12T17:53:11Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

k8s-ci-robot · 2020-07-12T21:07:07Z

@verb: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
pull-kubernetes-e2e-kind-ipv6	c1086c6c5174ef2ee8c183407bc1f54932a5956e	link	`/test pull-kubernetes-e2e-kind-ipv6`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

fejta-bot · 2020-07-12T23:08:12Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

k8s-ci-robot requested review from mengqiy and soltysh June 19, 2020 15:56

k8s-ci-robot added the area/kubectl label Jun 19, 2020

verb force-pushed the cli-debug-node+aylei branch from 1c3bcc5 to c4398ad Compare June 19, 2020 16:03

tedyu reviewed Jun 19, 2020

View reviewed changes

soltysh self-assigned this Jun 22, 2020

soltysh approved these changes Jun 22, 2020

View reviewed changes

verb force-pushed the cli-debug-node+aylei branch 2 times, most recently from d60f19b to c1086c6 Compare June 24, 2020 12:43

kubectl debug: add support for debugging nodes

7e63cc8

When called with a node target, `kubectl debug` will create a run-once pod in the target node's namespaces.

verb force-pushed the cli-debug-node+aylei branch from c1086c6 to 7e63cc8 Compare July 9, 2020 09:31

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 9, 2020

verb changed the title ~~WIP: kubectl debug: support debugging nodes~~ kubectl debug: support debugging nodes Jul 9, 2020

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 9, 2020

soltysh approved these changes Jul 9, 2020

View reviewed changes

k8s-ci-robot added this to the v1.19 milestone Jul 9, 2020

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 9, 2020

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 9, 2020

k8s-ci-robot merged commit 2234e2b into kubernetes:master Jul 13, 2020

github-actions bot mentioned this pull request Jul 14, 2020

Week Ending July 12, 2020 dev-obs/actus#193

Open

dinever mentioned this pull request Sep 17, 2020

Fix stderr usage in kubectl debug #94856

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

kubectl debug: support debugging nodes #92310

kubectl debug: support debugging nodes #92310

verb commented Jun 19, 2020

tedyu Jun 19, 2020

verb Jun 21, 2020

soltysh left a comment

soltysh Jun 22, 2020

verb Jun 23, 2020

soltysh Jun 22, 2020

verb Jun 23, 2020

soltysh Jun 22, 2020

verb Jun 23, 2020

soltysh Jun 22, 2020

verb Jun 23, 2020

verb commented Jul 9, 2020

soltysh left a comment

k8s-ci-robot commented Jul 9, 2020

fejta-bot commented Jul 10, 2020

verb commented Jul 10, 2020

fejta-bot commented Jul 10, 2020

fejta-bot commented Jul 11, 2020

fejta-bot commented Jul 11, 2020

verb commented Jul 11, 2020

fejta-bot commented Jul 11, 2020

fejta-bot commented Jul 12, 2020

joelsmith commented Jul 12, 2020

joelsmith commented Jul 12, 2020

fejta-bot commented Jul 12, 2020

fejta-bot commented Jul 12, 2020

fejta-bot commented Jul 12, 2020

k8s-ci-robot commented Jul 12, 2020 •

edited

fejta-bot commented Jul 12, 2020

kubectl debug: support debugging nodes #92310

kubectl debug: support debugging nodes #92310

Conversation

verb commented Jun 19, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

soltysh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

verb commented Jul 9, 2020

soltysh left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jul 9, 2020

fejta-bot commented Jul 10, 2020

verb commented Jul 10, 2020

fejta-bot commented Jul 10, 2020

fejta-bot commented Jul 11, 2020

fejta-bot commented Jul 11, 2020

verb commented Jul 11, 2020

fejta-bot commented Jul 11, 2020

fejta-bot commented Jul 12, 2020

joelsmith commented Jul 12, 2020

joelsmith commented Jul 12, 2020

fejta-bot commented Jul 12, 2020

fejta-bot commented Jul 12, 2020

fejta-bot commented Jul 12, 2020

k8s-ci-robot commented Jul 12, 2020 • edited

fejta-bot commented Jul 12, 2020

k8s-ci-robot commented Jul 12, 2020 •

edited