On November 5, Microsoft documented a setup that lets GitHub Copilot use Qwen3.8-27B with inference on the developer's computer. Foundry Local runs the model on an NVIDIA GPU, and the GitHub Copilot app sends requests to the local service configured as the provider. For this model variant, Microsoft lists a four-bit quantized file of about 18 GB and recommends at least 24 GB of video memory for the setup described.
How a request goes from Copilot to the local model
Foundry Local is the runtime that downloads, loads, and runs the model. In Microsoft's example, it starts a service with an API address on the user's computer. The GitHub Copilot app lets users add a model provider through Bring Your Own Key, abbreviated BYOK. Once the address returned by Foundry Local is configured, Qwen3.8-27B can be selected in the app's model picker. The address depends on the instance running on that computer.
The Copilot app handles the conversation and interaction with coding tools. Foundry Local receives the inference request and the GPU generates the model response. In a workflow involving a source repository, requests can include selected files, a description of a change, and test results. The model receives the context the client sends. Permissions for files, the terminal, and other tools remain under the control of the application and the environment the team uses.
The local route concerns generation by the model after installation and loading. The first download of the files is a separate operation. App sign-in, repository synchronization, and extensions have their own connections and rules. For a team working on a private project, the distinction is operational. The destination of the model request is visible in the provider configuration, while the paths taken by files used by the other tools are determined separately.
What a GPU with 24 GB or 32 GB of VRAM can load
Qwen3.8-27B is a dense model in the Qwen family with 27 billion parameters. The model authors list public weights under the Apache 2.0 license. Microsoft chose this model as the focus of its local coding optimizations and describes a four-bit variant. Quantization reduces the space taken up by weights. During inference, available memory is also used for context, the runtime's working data, and outputs generated in the session.
Microsoft says the described setup on an NVIDIA GPU with 24 GB of VRAM can use a context window of up to 64K tokens. With 32 GB or more, the company says the model's limit of about 262,000 tokens can be reached. Context measures the text units available for instructions, code, conversation history, and its continuation. The way the application selects which files to put into the request also matters.
Foundry Local uses ONNX Runtime for execution. For Qwen3.8-27B, Microsoft worked on two distinct stages of inference. The first processes the input, including the files and history it receives. The second generates the continuation. The engine can preserve calculations for the common beginning of successive requests and group requests that arrive while it is already working. These techniques matter when a task revisits the same code after each test result.
In the generation stage, Microsoft describes DFlash2, a method in which a smaller component proposes multiple tokens for verification. The company also describes quantizing context memory and moving some data into memory accessed by the CPU. These options reduce pressure on video memory in the documented setups. Their effect depends on the runtime settings and the size of the request being processed.
The example uses Foundry Local CLI 0.13.0 or later. Microsoft specifies that CUDA and compatible cuDNN libraries must be installed separately for execution on NVIDIA hardware. They are requirements of this Qwen3.8-27B setup. Foundry Local is available across several platforms, and Microsoft describes extending this specific Qwen optimization to Macs and AMD GPUs as future work.
Microsoft's tests and the cost of a local session
Microsoft compared Foundry Local and llama.cpp using four-bit Qwen3.8-27B on two systems. One had an NVIDIA GeForce RTX 5090 with 32 GB of VRAM. The other was an NVIDIA DGX Spark with 128 GB of unified memory. The evaluation used data from GitHub issues. Microsoft reports faster generation at the input lengths tested and a wider gap beyond about 131,000 tokens under that protocol.
The two systems use different types of memory. The RTX 5090 has its own video memory, while the processor and graphics on the DGX Spark share system memory. Microsoft's charts show the result for each platform separately. The DGX Spark's 128 GB describes total unified memory in the test configuration, and the RTX 5090's 32 GB describes memory on the graphics card.
The comparison measures generation speed under Microsoft's test conditions and configurations. A complete workflow also includes preparing the context, calling tools, running tests, and reviewing changes. A model used to explain a file of a few hundred lines and the same model used to examine a large repository can place different demands on memory and the GPU.
In this integration, the computation for tokens generated by Qwen takes place on the team's workstation. The resources involved include the hardware available, the electricity it uses, and maintenance of the setup. The team can also use other providers in the app according to the features available in the client. For development on internal code, a concrete aspect of the new documentation is the ability to keep the same working client and explicitly select the provider that runs the model.
