Can an AI model work without the internet?
Yes: a compatible downloaded model can generate answers on your computer without sending each prompt to a remote model. You need the software and model files first. A local text model does not automatically include web search, current news or access to your documents.
Ollama is one available way to run downloaded models on macOS, Windows and Linux. Its app also offers cloud models, so installing a desktop app is not enough to establish that a particular conversation is local. This guide uses Ollama's documented workflow as an example, rather than a claim that it is the only suitable tool.
Start with one narrow task, such as extracting action items from fictional notes. A short, inspectable task makes it easier to compare accuracy, speed and memory use before handing the model a large folder. The goal is to verify a working setup on your machine, not to buy hardware from a generic specification.
What do RAM, VRAM and model size tell me?
RAM is system memory; VRAM is memory available to a graphics processor. On unified-memory machines, CPU and GPU share a memory pool. Hardware support depends on the operating system, GPU and driver, so check Ollama's current compatibility page for the exact device instead of assuming any graphics card will accelerate a model.
Model weights hold learned numerical parameters. Quantization stores those values with reduced precision to lower their storage requirement, potentially changing answer quality. A smaller download is not automatically the best choice for your task. Compare the same questions with an answer key.
Here is a deliberately simplified memory calculation: eight billion parameters × four bits ÷ eight bits per byte = four billion bytes, or 4 GB in decimal units, approximately 3.73 GiB. This accounts only for hypothetical weight values stored at exactly four bits each.
It is not a laptop purchase recommendation or an exact file size. Quantization metadata, values stored at other precisions and the program's working memory add requirements. The operating system and other apps also need space. Use the actual model download size and measured runtime usage, leaving headroom rather than treating that 4 GB calculation as a fit guarantee.
Why can a longer conversation use more memory?
The context window is the amount of tokenized material the model can access during a request. A larger configured context increases memory requirements. A model that handles a short question comfortably can struggle with a large document or a long conversation.
Ollama exposes a context setting in its app. Check the value in your installation instead of assuming one default for every machine and release. With a model running, the command ollama ps reports loaded models, the CPU/GPU split and allocated context in current versions.
For an initial comparison, keep the prompt, output request and context setting unchanged. If a run fails for lack of memory, reduce the task or context before concluding that the model is unusable. If you change both the model and context at once, the result will not tell you which change mattered.
Set up one downloaded model and confirm the route
Install Ollama using its official instructions for your operating system. In the model selector, choose a downloadable local model compatible with your available memory and intended task. Read its model card and license, record the exact model name or tag, and allow the download to finish while online.
For the command-line route, ollama ls lists available downloaded models and ollama ps lists running ones. Use ollama run followed by the exact local model tag you selected to start a conversation. Do not copy a cloud-model example when your goal is offline execution.
Ollama's FAQ documents a local-only control: set disable_ollama_cloud to true in its server.json configuration, preserving other settings, or configure OLLAMA_NO_CLOUD=1 for the server process using the instructions for your operating system. Restart Ollama after the change and confirm the cloud-disabled message in its logs. Setting a variable in an unrelated terminal may not affect an app already running.
Keep the default local service binding for this exercise. Do not expose the service to a public network or attach web tools just to make a first chat work. Cloud disabling removes Ollama cloud features; it does not control every extension or separate application you might connect.
Five-minute offline test with an answer you can check
Prepare this invented note: “Workshop: Thursday, 14:00–15:00. Mira will send the slides by Wednesday. Joel will confirm the room by Tuesday. No ticket price has been decided.” Ask for the two owners and deadlines, the workshop time, and the ticket price. Use the prompt below.
After downloading and loading the local model, disconnect the computer's network connections using your normal controls. Start a new conversation and run the note again. Record whether generation finishes. A previous cached answer on screen is not an offline test.
The answer key is: Mira — slides — Wednesday; Joel — room — Tuesday; workshop — Thursday, 14:00–15:00; ticket price — not specified. Check each field separately. If it invents a currency amount, that is an accuracy failure even if the model ran entirely offline.
Then change only the workshop start to 14:30 and rerun in a fresh conversation. A correct revision is 14:30–15:00, with the two deadlines unchanged. This controlled change makes it easier to detect a stale result or a model that rewrites facts casually.
Record the model tag, context setting, device, first-response wait and total completion time using the same method each time. The first run can include loading work, so label it separately. This is a test protocol for you to perform; no laptop benchmark or successful model execution is claimed here.
PROMPT TO ADAPT
Use only these fictional notes. Extract owner, action and deadline for each assignment. Then give the workshop day, start and end time, and ticket price. Write “not specified” for missing facts. Do not browse, guess a date or invent a price. Notes: Workshop: Thursday, 14:00–15:00. Mira will send the slides by Wednesday. Joel will confirm the room by Tuesday. No ticket price has been decided.
Does local mean private, free or always correct?
Ollama states that it does not see prompts when models run locally. Its cloud models process prompts and responses remotely. That distinction concerns where inference runs; a separate chat app, plugin, backup service or log file may have its own data handling. Review the whole workflow before using sensitive documents.
A successful offline answer demonstrates that this tested path could generate without a live network connection. It does not prove the software never stores information or will never transmit it after reconnection. Use fictional input for the test and inspect settings before introducing real private material.
Local inference still uses electricity, disk space, memory and your time. Model licenses can impose conditions, and useful speed depends on the actual workload. A cloud option can offer models your laptop cannot hold, but introduces remote processing and provider-specific usage terms. Compare those tradeoffs using your task rather than assuming one route is universally cheaper.
Local models can produce unsupported claims just like remote ones. If you need answers grounded in a document collection, use the RAG guide to plan retrieval and evidence checks. If you need current prices or news, an offline model's confident answer is not a verified live source.
Troubleshoot before changing hardware
If the model will not start, verify that the model files finished downloading and that its type is supported by the installed runtime. If it reports insufficient memory, close unnecessary apps, try a smaller compatible model or reduce context. Record which variable changed.
If it works online but fails offline, check whether you chose a cloud model or enabled a tool that needs the internet. If answers are slow, inspect the CPU/GPU split and compare like-for-like prompts; do not treat the computer's advertised GPU name as proof it was used.
Finish with a small acceptance log: local model identified; fresh offline answer completed; four fields checked against the key; memory and timing recorded; unresolved issues listed. Only then expand the task. Recheck after changing the model, runtime or integrations, because the earlier result covers the earlier setup.
- The exact local model and its license are identified.
- Model download is complete before disconnecting.
- Cloud processing and external tools are distinguished from local inference.
- The fictional answer is verified, not accepted because it sounds fluent.
- Observed results are recorded without claiming a privacy audit or benchmark.
Sources & further reading
- Ollama: official setup guide
- Ollama: hardware support
- llama.cpp: quantization and its tradeoffs
- Ollama: context length and memory
- Ollama: local mode, cloud controls and configuration
- Ollama: command-line reference
- Ollama: cloud models and data handling
Educational content. Examples are illustrative. Protocols, products and rules can change; consult the linked documentation for current details.