Utilizing Online LLMs from Terminal

tforumians,

This guide is going to cover how to access and utilize online LLMs from your terminal (no JavaScript required!). We will be focusing on DuckDuckGo as our primary provider.

You may be asking yourself, 'Why would I use an online LLM rather than a local LLM'? Well, the answer boils down to ease of access, speed, and/or your available computing resources. For example, you might not want to download a > 5 GB model, or you don't have the recommended specs to interface with a particular model (and achieve reasonable throughput). So, for quick and easy LLM communication (while maintaining a semblance of privacy: https://duckduckgo.com/aichat/privacy-terms), tgpt does not fall short.

Now with all of that out of the way, let's jump in.

Terminal GPT (tgpt)

tgpt is a cross-platform command-line interface (CLI) tool that allows you to use AI chatbot in your Terminal without requiring API keys.

– https://github.com/aandrew-me/tgpt

1. We will need to download and install the tool by running the following command:

$ curl -sSL https://raw.githubusercontent.com/aandrew-me/tgpt/main/install | bash -s /usr/local/bin

　

2. Once installed, we will set our provider environment variable to DuckDuckGo:

$ export AI_PROVIDER='duckduckgo'

　

3. In addition, there are four models to choose from:

　

Model

gpt-4o-mini (default)

claude-3-haiku-20240307

meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo

mistralai/Mixtral-8x7B-Instruct-v0.1

　

To select a model other than the default 'gpt-4o-mini', we will use the `--model` flag:

$ tgpt --model 'claude-3-haiku-20240307' [prompt]

　

4. There are also three different input options you can use:

　

Inline is passing your prompt in the same initial command (i.e. no flag necessary)
Interactive is how it sounds, except is limited in scope when it comes to free-form writing (`-i` or `--interactive`)
Multiline is the most robust, and will probably be your 'default' option (`-m` or `--multiline`)

$ tgpt --model 'meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo' [-i|-m] [prompt]

　

5. Now that we know the basics, let's pass it a prompt:

$ tgpt --model 'mistralai/Mixtral-8x7B-Instruct-v0.1' 'Are you familiar with the darknet forum, tforum?'

　

That about sums it up! (It's also worth noting that tgpt can be used as an interface with Ollama as well.)

　

Lastly, if you have any questions while using tgpt, make sure to use the 'help' option:

$ tgpt -h, --help

Amateru 天照