Migrate Terraform commands from `cli.Ui` to `views.View` abstractions when rendering output. Investigate use of `Streams`, too.

[SarahFrench]

Background

Different parts of the code base render output using either the older cli.Ui abstraction, or the newer views.View abstraction.

Views were introduced in #27738 to minimise the code needed to render output in either Human or JSON formats. Since then, only commands that have the option to create JSON/machine-readable output have been migrated to using Views. Other commands are still using the old Ui approach, and it's possible for one command to use a mixture of the two, due to both being accessible in a command's Meta. Two parallel solutions for the same problem makes it harder to onboard to the codebase.

Proposal

We should ensure that we migrate all commands over to using Views eventually. This appears to have been the plan in the past, but understandably it's a low-priority piece of work:

terraform/internal/command/meta.go

Lines 635 to 636 in eee744c

	// Reconfigure the view. This is necessary for commands which use both
	// views.View and cli.Ui during the migration phase.

Not in scope

• Changes to how input is gathered.
• Eliminating use of github.com/hashicorp/cli

Progress

Command	Uses arguments package?	Uses views package?	Full migrated?	PRs
console	Yes	No
fmt	Yes	No
get	Yes	Yes	✅	#39217
graph	Yes	No
import	Yes	No
init	Yes	Yes	✅	#37920 <- removed use of cli.Ui in meta_backend.go
login	Yes	No		Args: #39319
logout	Yes	No		Args: #39318
providers	Yes	No
providers lock	Yes	No
providers mirror	Yes	No
providers schema	Yes	No
state identities	Yes	No		Args: #39334
state list	Yes	No
state pull	Yes	No
state push	Yes	No
state rm	Yes	No
state mv	Yes	No
state replace-provider	Yes	No
taint	Yes	No
unlock	No	No		Args: #39336
untaint	Yes	No
version	Yes	Yes
workspace new	Yes	Yes	✅	#39191
workspace list	Yes	Yes	✅	#38925
workspace select	Yes	No
workspace show	Yes	Yes	✅	#39198
workspace delete	Yes	No

Also, an honourable mention for code in internal/command/meta_backend.go that mixes Views and Ui. That code is specific to the init command though, so could be addressed at the same time as the init command.

Caveats

Both use of cli.Ui and views logs errors to stderr and warnings to stdout. See: #38811

References

Code

The Meta allows access to both a Ui and a View:

terraform/internal/command/meta.go

Lines 75 to 79 in eee744c

	View *views.View
	Color bool // True if output should be colored
	GlobalPluginDirs []string // Additional paths to search for plugins
	Ui cli.Ui // Ui for output

The Meta struct also has some methods for handling output via Ui:

terraform/internal/command/meta.go

Line 684 in eee744c

func (m *Meta) showDiagnostics(vals ...interface{}) {

The newer alternative(s) of the method above is the Diagnostics method in the Operation interface which is implemented differently for Human or JSON output:

terraform/internal/command/views/operation.go

Lines 24 to 37 in eee744c

	type Operation interface {
	Interrupted()
	FatalInterrupt()
	Stopping()
	Cancelled(planMode plans.Mode)
	EmergencyDumpState(stateFile *statefile.File) error
	PlannedChange(change *plans.ResourceInstanceChangeSrc)
	Plan(plan *plans.Plan, schemas *terraform.Schemas)
	PlanNextStep(planPath string, genConfigPath string)
	Diagnostics(diags tfdiags.Diagnostics)
	}

Related issues

Potentially addresses #34768

[SarahFrench]

A consideration with this migration is whether HCP Terraform parses 'human' output from the unmigrated commands currently, and how confident we are to consider the output via Ui and output through a View in a OperationHuman equivalent. We probably want to work with the Core Cloud team to ensure this issue's changes doesn't cause any integration issues, if we do this work.

[SrushtiSapkale]

Hi! I’d like to work on this issue. Could you please share some pointers or guidance on how to get started? Thanks!

[crw]

@SrushtiSapkale Please see https://github.com/hashicorp/terraform/blob/main/.github/CONTRIBUTING.md.

[SarahFrench]

For anyone interested in this issue, please note that it will require the reviewer to work with other teams internal to HashiCorp before merging the changes.

A consideration with this migration is whether HCP Terraform parses 'human' output from the unmigrated commands currently, and how confident we are to consider the output via Ui and output through a View in a OperationHuman equivalent. We probably want to work with the Core Cloud team to ensure this issue's changes doesn't cause any integration issues, if we do this work.

[SarahFrench]

Another thing I've found: Some commands may use Streams to render output, such as the state show command. It's also used in test, graph and others.

I haven't investigated Streams yet, and why one may be used instead of Views (which has streams internally).

terraform/internal/cloud/test.go

Lines 98 to 101 in 808713f

	// View and Streams provide alternate ways to output raw data to the
	// user.
	View views.Test
	Streams *terminal.Streams

[apparentlymart]

FWIW I remember the original intention here was for terminal.Streams to deal with low-level problems like whether the output is a terminal at all and how wide it is if so, but that the views concept should be how commands should interact with those details, rather than accessing them directly.

IIRC a few commands did get rewritten from using c.Ui to c.Streams directly as an intermediate step because github.com/mitchellh/cli was no longer maintained and the decision hadn't yet been made to fork it as github.com/hashicorp/cli. My understanding was that the intention had always been to use views consistently in the long run though, and it was just that the status of the mitchellh/cli library created enough pressure to think about moving away from it even though there wasn't the resources to do full structuring with views at the time.

---

You're right that there's a missing piece here for accepting input. The Streams API has the lowest level part of this in its handling of stdin, but we didn't have a great answer for what "views-level" abstraction to wrap around it, because the initial priority was mainly about being able to generate JSON output and there's never any interactive input in that mode.

Note also that the interactive mode of terraform console doesn't gather input through c.Ui, and instead uses the third-party library github.com/chzyer/readline. I can't look at the Terraform code anymore but I looked at the equivalent code in another similar codebase and found that it's still using os.Stdin/etc directly instead of using the equivalents accessible through terminal.Streams, so unfortunately that seems like another entirely-separate strategy unlike how any other command is implemented. 😖

[SarahFrench]

Thanks @apparentlymart! That context about mitchellh/cli => hashicorp/cli is really useful. Hopefully we can get some movement back in the migration to Views in the near future.