general doc/highlights from ai during building - twily.info 2026

---

if issues starting ollama, may need to update your graphic driver

also may need to update ollama to support gguf files uncensored

to start from scratch:

need python/pip pytorch etc

need stable diffusion 1.5 for image gen

need ollama for reasoning text gen

webfiles does not need to be on same server running the AI

change BACKEND ip in setup.php

all web files are index.php (vanilla js) and ajax.php

setup with apache2 or nginx and php-fpm 
(increase php max execution time to 1800 in php.ini,
enable php extension 'gd' in php.ini,
add Timeout 1800 to bottom of apache2.conf if using apache,
see fcgi or proxy timeout for nginx below)

setup_and_launch is for AI backend computer, only includes ollama

---

1. Hardware Budget & Model Strategy for GTX 1070 (8GB VRAM)

An 8GB GTX 1070 is a brilliant vintage workhorse, but it has strict VRAM limits. To keep things fast and prevent Out-Of-Memory (OOM) errors:

    Text Models (Ollama):

        Stick to 1.5B to 8B quantized models (Q4_K_M or Q5_K_M).

        TwIL-8B / Qwen3 / DeepSeek distillations running via Ollama at 4-bit or 5-bit quantization will fit comfortably in VRAM while maintaining solid reasoning speed. Avoid anything above 8B unless offloading layers to RAM (which tanks speed).

    Image Generation (Stable Diffusion):

        Forge UI or AUTOMATIC1111 configured with --medvram or --lowvram flags.

        SD 1.5 variants / Anime checkpoints (like Anima Light Lavender): These fly on a GTX 1070 (generating 512x512 or 768x768 images in seconds).

        SD 3.5 Medium: Might push the 8GB VRAM limit hard due to its larger text encoders and architecture. If you try it, ensure you run it with optimized attention (--opt-sdp-attention) or stick to lighter weights.

---

3. Structuring Flat-File Storage

To match your ideal workflow where each chat has its own JSON ledger and assets folder:
Plaintext

/ai-chat/
├── index.php
├── ajax.php
└── chats/
    ├── chat-1712684800.json          <-- Main conversation log (array of messages)
    ├── chat-1712684800/
    │   ├── generated/
    │   │   └── img-1712684850.png    <-- Saved generated images
    │   └── uploaded/
    │       └── source-1712684900.png <-- User uploaded images for multimodal input
    └── chat-1712690000.json

---

Part 1: Windows Python Setup & Launcher Script (setup_and_launch.py)

Run this script on your Windows 10 machine. It handles setting up a Python virtual environment, installing dependencies, pulling models (like LLaMA 3.2 / Qwen 1.5B or similar lightweight reasoners fitting an 8GB card), and spinning up the servers with LAN binding (0.0.0.0) so your Linux PHP server can talk to them.

The Python Setup Script (setup_and_launch.py) for Windows to build the environment, pull lightweight models suitable for an 8GB GTX 1070, and launch them with API/LAN access flags.

---

1. Maintaining and Updating the Forge venv & Pip

Because Stable Diffusion WebUI Forge embeds its own isolated Python environment, you shouldn't mess with your global Windows Python installation. Instead, target Forge's local virtual environment directly.

To safely update pip and maintain dependencies inside your Forge folder (F:\stable-g2\stable-diffusion-webui-forge), open a command prompt there and run:
DOS

cd /d F:\stable-g2\stable-diffusion-webui-forge

:: 1. Upgrade pip inside Forge's internal venv
venv\Scripts\python -m pip install --upgrade pip

:: 2. If you need to update core packages (like torch/transformers if required by extensions)
:: Note: Only do this if an extension breaks; otherwise, leave PyTorch locked to what Forge shipped with.
venv\Scripts\python -m pip install --upgrade requests urllib3

3. Recommended Startup Flags for your GTX 1070

When launching webui-user.bat, make sure your arguments look like this to optimize your 8GB VRAM and open up LAN access for your Linux PHP server:
DOS

set COMMANDLINE_ARGS=--medvram --api --listen --opt-sdp-attention
call webui.bat

    --medvram: Keeps heavy model parts off your system RAM while saving VRAM.

    --api: Enables the JSON endpoints your PHP script relies on.

    --listen: Binds to 0.0.0.0, letting your Linux web server across the LAN send jobs straight to your Windows rig.

---

1. Fix Ollama for API & LAN Access (Windows)

    Close the interactive prompt (/bye or close the cmd window where you ran ollama run).

    Ollama on Windows runs as a background tray application. To make sure it accepts requests from your Linux server (10.0.0.64), you need to set the environment variable for binding:

        Press Win + R, type sysdm.cpl, go to Advanced -> Environment Variables.

        Under User or System variables, click New:

            Variable name: OLLAMA_HOST

            Variable value: 0.0.0.0:11434

        Restart the Ollama application from your Windows system tray.

    Verify it's ready by running this from your Linux server shell to test the API route directly:

curl http://10.0.0.89:11434/api/generate -d '{"model": "qwen2.5:7b-instruct-q4_K_M", "prompt": "test", "stream": false}'

---

1. Kill and Restart Ollama Cleanly on Windows

    Open Task Manager (Ctrl + Shift + Esc).

    Look for Ollama or ollama app under processes, right-click it, and select End Task.

    Go to your Windows system tray (bottom right corner near the clock), right-click the Ollama icon if it's still lingering there, and choose Quit.

    Open a fresh Command Prompt (cmd) and verify that the environment variable is visible to the shell:
    DOS

echo %OLLAMA_HOST%

---

Fixing Ollama CORS on Windows

    Quit Ollama: Right-click the Ollama icon in your Windows Taskbar system tray and choose Quit.

    Set the Environment Variable:

        Open your Windows Start menu, search for Environment Variables, and click Edit the system environment variables.

        Click the Environment Variables... button.

        Under User variables (top half), click New...

            Variable name: OLLAMA_ORIGINS

            Variable value: *

        Click OK on all windows to save.

    Restart Ollama: Launch Ollama again from your Start Menu so it reads the new environment configuration.

---

2. Setting Up an Uncensored Hugging Face Model in Ollama

Directly running ollama run hf.co/... works for standard models hosted on Hugging Face, but for community GGUF builds (like mradermacher's GGUF repos), you can pull it cleanly using a local Modelfile or direct GGUF import:

    Download the GGUF file (e.g., a Q4_K_M or Q5_K_M quantization to fit your VRAM comfortably) from the Hugging Face repo to your local machine.

    Create a file named Modelfile in that same directory with a single line:
    Dockerfile

FROM ./DeepSeek-V4-Pro-Qwen3.5-9B-UNCENSORED-heretic.gguf

Register and build it locally inside Ollama via terminal:
Bash

ollama create deepseek-uncensored -f ./Modelfile

Test or run it immediately:
Bash

ollama run deepseek-uncensored

(replace deepseek with gemma4 by trevorjs for lighter model)

(diffuse model is separate all together, use the engine
'stable-diffusion-webui-forge' best for generating,
get 'kohya_ss' if looking to train data sets)

(get checkpoint models from civitai for 1.5 pony-)

---

How to Import a Raw Safetensors Model into Ollama

    Keep all those files downloaded together in a dedicated local folder (e.g., ./models/DeepSeek-V4-Pro-Qwen3.5-9B/), including config.json, the 4 model-0000x-of-00004.safetensors files, and tokenizers.

    In that same folder, create your Modelfile containing the folder path (or ./ if Ollama is run from inside that directory):
    Dockerfile

FROM ./

Build and register it in Ollama from your terminal:
Bash

ollama create deepseek-uncensored -f ./Modelfile

---

file upload: text/code + prompt to reason ai, image + prompt to diffuse ai, image + prompt to vision ai
/imagine command to use diffuser, reason vs vision is detected by having files uploaded be of image or text kind

vision model (+8GB): ollama pull llama3.2-vision
or current default because mllama architecture error:
vision model (+8GB): ollama pull qwen2.5vl:7b
(see 'alternative' below but use same Makefile as here)

make file similar to gemma: Makefile-vision
with the content to increase context similarly:
===
FROM llama3.2-vision
PARAMETER num_ctx 32768
===

and run

ollama create llama3.2-vision -f Modelfile-vision

in same folder used for gemma4, then restart as normal:

ollama run gemma4-uncensored

(do not need to run the vision in a second terminal, just run the main reasoning model)

---

if vision model not working because mllama architecture:

Update your local Ollama installation to the latest release so its runtime engine supports mllama architectures:
Bash

curl -fsSL https://ollama.com/install.sh | sh

or windows:

winget install -e --id Ollama.Ollama

to update only

winget upgrade -e --id Ollama.Ollama

alternative if all else fails to get llama3.2-vision:
(which it did for me so code reflect this)

ollama pull qwen2-vl
or
ollama pull qwen2.5vl:7b

and same process in create with ctx Makefile-

ollama cleanup:

ollama list
ollama rm <model>
ollama stop <model> (same as /bye within)

---

Stubborn ollama / mllama architecture error:

Make sure kill all daemons of ollama, task manager etc

taskkill /f /im ollama.exe
taskkill /f /im ollama_llama_server.exe

and start cleanly winget version

ollama serve

then proceed the ollama run command in a second terminal

(if ollama is installed as a native windows service
Stop-Service Ollama
Stop-Process -Name "ollama*" -Force -ErrorAction SilentlyContinue
or services.msc)

---

(place gguf file in same directory)
Makefile content for gemma4 uncensored gguf :
===
FROM ./gemma-4-E2B-it-uncensored-Q8_0.gguf
PARAMETER num_ctx 32768
===

num_ctx: fix for default ollama starts with tiny context window, making the ai leave a "no response" on large query/big contexts/files etc

---

if using nginx reminder each site-enabled needs
client_max_body_size 64M
to upload more than 1M

nginx relevanse for php timeout 504 errors:
    location ~ \.php$ {
        fastcgi_read_timeout 300s;
        fastcgi_send_timeout 300s;

or
    location / {
        #proxy_read_timeout 300s;
        #proxy_connect_timeout 300s;
        #proxy_send_timeout 300s;

---

php.ini also has file upload size limit, relevant variables:

max_execution_time = 1800
max_input_time = 300
memory_limit = 256M

post_max_size = 5050M
upload_max_filesize = 5000M

file_uploads = On
max_file_uploads = 50
default_socket_timeout = 60

---

ai note* native PHP extensions like cURL do not respect default_socket_timeout. If your PHP script connects to an AI API via cURL and you haven't explicitly set a cURL timeout, PHP will wait forever.

Best Practices for AI Chat Sites
Explicitly cap cURL: Always set an explicit timeout in your cURL options slightly below your Nginx timeout (e.g., 240 seconds) so PHP handles the failure gracefully rather than letting Nginx murder the request

curl_setopt($ch, CURLOPT_TIMEOUT, 240);       // Max total execution time for the AI response
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);  // Max time just to establish the connection