Summary

AIFunctionBuilder takes a prompt from the user and generates a PowerShell function which is validated for syntax and logical issues so you don’t have to do the boring work.

Getting Started

• Watch the video below for a summary of what the module can do.
• The function builder comes with PowerShell AI which can be installed in a PowerShell terminal with the command:
  Install-Module PowerShellAI
• To launch the function builder just enter the command Invoke-AIFunctionBuilder with no parameters.

How it Works

I’ve spent a bit of time spent going back-and-forth between ChatGPT and other LLM tooling to validate any output that’s more complicated than a one-liner.

The approach I’ve built here is an experiment that uses an automated feedback loop to iterate on the initial script the LLM produces so I don’t have to muck around fixing the obvious issues. By utilizing PSScriptAnalyzer and the PowerShell Abstract Syntax Tree (AST) to parse the script and check for the common signs of hallucination the script can be validated to a degree that should provide much better reliability on the first execution.

The script is not executed until the user reviews it because blindly executing code can cause some damage (think: write me a script to clean up space on my harddrive).

I chose to focus this solution on “functions” instead of PowerShell code in a general sense because they’re relatively easy to reason about, they generally take parameters and return values. Realistically this is producing a single-function module but as seen in the video above it can also produce multi-function modules occasionally.

When models can consume more context you will be able to tell the model to take most of the checks I’ve detailed below into consideration before it spits out some code but realistically these GPT models are predicting a stream of words and are very general purpose, there will always be parsers in the mix because they have the upper hand when it comes to validating the correctness of any output.

This is a rough view of the algorithm in the function builder and some of the syntax validations performed to make the LLM generate a decent quality output:

• 🧠 Ask the LLM to try write a function to meet my requirements.
• ✅ No missing parentheses, brackets, unclosed strings etc.
• ✅ Commands are available.
• ✅ Parameters used in the script are correct for each commandlet.
• ✅ No ambiguous unnamed parameters are used.
• ✅ No named parameters are being used multiple times on the same command.
• ✅ At least one parameter set is satisfied by the parameters provided.
• ✅ Types used by New-Object/Add-Type are real and resolvable.
• ✅ All .NET static methods used exist and have the correct number of params.
• ✅ All .NET constructors have the correct number of parameters passed to them.
• ✅ All .NET static properties used exist.
• 🧠 Ask the LLM “given the prompt {A} does the function {B} meet all requirements?
• 🧠 If not, fix the logical issues”.
• ➰ Repeat the above checks until no syntax issues or logical failures are present.