foldr me 0 universe

Xiaoyi Chen

Multimodal LLM has brought LLM to new dimensions. When LLMs can see, their capabilities multiply. GPT-4V(ision) produced some impressive demos, including understanding jokes in images, acting as e-sports commentator, turning mock ups into apps and reproducing pictures in novel styles.




What if you can have a GPT-4V like model running on your laptop? [LLaVA](https://llava-vl.github.io/) is an Open Source multimodal model similar to GPT-4V, reaching 85.1% of relative score in visual chat compared with GPT-4V. Here's how to get LLaVA up and running on macOS.



<figure align="center">
    <img src="/media/llava-webui-example.png"  />
    <figcaption>LLaVA WebUI </figcaption>
</figure>

## Installation

Prerequisites: Install [miniconda](https://docs.conda.io/projects/miniconda/en/latest/).

```shell
mkdir -p ~/miniconda3
curl https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-arm64.sh -o ~/miniconda3/miniconda.sh
bash ~/miniconda3/miniconda.sh -b -u -p ~/miniconda3
rm -rf ~/miniconda3/miniconda.sh
```

Follow [LLaVA installation instructions](https://github.com/haotian-liu/LLaVA/blob/main/docs/macOS.md):

Clone the repository:

```shell
git clone https://github.com/haotian-liu/LLaVA.git
cd LLaVA
```

Install Python dependencies:

```shell
conda create -n llava python=3.10 -y
conda activate llava
python -mpip install --upgrade pip  # enable PEP 660 support
pip install -e .
pip install torch==2.1.0 torchvision==0.16.0
pip uninstall bitsandbytes
```

## Usage

You can either use Web UI or CLI for inference. You'd need to add `--device mps` for running model worker or CLI on Apple silicon.

**Web UI:**

First start controller, gradio web server, and start a model worker, then go to http://localhost:7860/. You can either specify `liuhaotian/llava-v1.5-13b` as `--model-path` (models are downloaded to `~/.cache/huggingface/hub/`), or download models from https://huggingface.co/liuhaotian/llava-v1.5-13b and specify the local model path.

Sometimes I got the error `"NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE."` on the Web UI, and model worker crashes. Restarting model worker would resolve this issue.

```shell
# activate conda where you start controller, web server or model worker
conda activate llava

# gradio controller and web server
python -m llava.serve.controller --host 0.0.0.0 --port 10000
python -m llava.serve.gradio_web_server --controller http://localhost:10000 --model-list-mode reload

# launch model worker
python -m llava.serve.model_worker --host 0.0.0.0 --controller http://localhost:10000 --port 40000 --worker http://localhost:40000 --device mps --model-path liuhaotian/llava-v1.5-13b
```

**CLI**:

Substitute `--image-file` with your image URL or path to local file.

```shell
conda activate llava
python -m llava.serve.cli \
    --model-path liuhaotian/llava-v1.5-7b \
    --image-file "https://llava-vl.github.io/static/images/view.jpg" \
    --device mps
```

## Ending notes

An even easier way if you just want to spin up the Web UI is to use [llamafile](https://github.com/mozilla-Ocho/llamafile), the latest hotness which compresses LLMs into a single executable with local web server or CLI. Follow [Simon Willison's guide](https://simonwillison.net/2023/Nov/29/llamafile/) here and it's a matter of downloading and executing a single file executable to run LLMs locally.

 Running LLaVA, an Open Source GPT-4V alternative on a Mac with Apple Silicon

## Introduction

Introducing "CaveTalkFun", the world's first programming language designed for cavemen. It uses natural language conversations between characters to represent functions.

![](/media/cavetalkfun/intro.png)

## Painter

Here's how you do "Hello world" in CaveTalkFun:

![](/media/cavetalkfun/cavepainter.png)

## Repeater

Need to do something over and over? Tell the repeater.

![](/media/cavetalkfun/repeater.png)

## Mapping a skill over a list of rocks

You can compose cavemen! Just hand the skiller to a magician:

![](/media/cavetalkfun/mapper.png)

## Future features

More exciting features coming to CaveTalkFun including Multi-modal Input, Contextual Assistance, Intuitive Visualization, Prehistoric-themed Libraries, and Interactive Learning!

![](/media/cavetalkfun/future.png)

CaveTalkFun - Prehistoric functional programming

> Can you write docs for the **public** APIs?

I vividly recall this feedback on my 2k-line PR with a deadline for reviewing. Over the years, it's a piece of advice that sticks as a surprisingly simple, but amazingly effective way to improve codebase quality.

Documenting public APIs compels you to reify and clarify your abstractions. The emphasis on **public** is important, as you are articulating your abstractions in a human-interpretable way. They are co-locating with your code, saving the trips to doc sites.

Considering the cost of writing and maintaining docs up to date might give you pause. But the time invested in writing would turn out to be trivial, compared to the time saved for fellow engineers on your team, or even yourself, staring blankly at the code and brain interpreting it, 20-level deep in a debugger stack, or frantically slack DMing. The values compound over time. Code is not a write-only medium. It's written for humans to read (and language models too). The more you write, the better you become at writing it.


<figure align="center">
    <img src="/media/human-reading-code.png"  />
    <figcaption>Human reading code by Stable Diffusion</figcaption>
</figure>

What about maintenance? Don't you need to maintain the stale docs if they get out of date? Well, if you find the docs get stale frequently, it's usually a sign that you have the wrong abstractions. Think about refactoring the abstractions instead. It's easy to delete the doc if they can't keep themselves up to date. Being [easy to delete](https://programmingisterrible.com/post/139222674273/write-code-that-is-easy-to-delete-not-easy-to) is another sign that docs are good code. Docs are [meta languages](https://frantic.im/meta-language/), but they are the types of meta languages that exist as code. They can be easily incorporated into code should you find the necessity.

In practice, docs could sometimes look like a verbatim of function signatures, which appears to contradict the conventional wisdom of ["Comments should not duplicate the code"](https://stackoverflow.blog/2021/12/23/best-practices-for-writing-code-comments/#:~:text=Rule%201%3A%20Comments%20should%20not%20duplicate%20the%20code,-Many%20junior%20programmers&text=While%20this%20may%20be%20a,time%20to%20write%20and%20read). I've explained above how this wisdom doesn't apply to writing docs for public APIs, in terms of writing and maintaining the docs. The argument against visual clutter doesn't hold up since you can fold docs easily in modern IDEs. On the contrary, they give you additional context when you use the interfaces.

A simple way to improve codebase quality

I recently was fascinated by Raspberry Pi and the maker communities behind it. One of the useful projects you can build with Raspberry Pi is [Pi-hole](https://pi-hole.net/). It blocks ads, tracking and malicious websites on every device on your local Wi-fi network, and have an easily manageable web interface.

_Caveat: Pi-hole could not block Youtube Ads. Your best bet would be using a browser like [Brave](https://brave.com/) (Founded by Brendan Eich, creator of JavaScript) or paying for Youtube premium._

## What you will need

- A [Raspberry Pi](https://www.raspberrypi.com/), for example Raspberry Pi 4 Model B
- Wi-Fi Router

## Connect Raspberry Pi to your local network

First, you would need to connect Raspberry Pi to your local network using ethernet cable or Wi-fi network. Additionally, you might want to enable SSH access on your Pi.

To connect to network:

- If you have a monitor, [connect to the monitor using an HDMI cable](https://www.raspberrypi.com/documentation/computers/getting-started.html#connecting-a-display), and [connect to your network from the Desktop GUI](https://www.raspberrypi.com/documentation/computers/configuration.html#using-the-desktop).
- Alternatively, you can use Raspberry Pi without a monitor (headless mode), and [connect from command line](https://www.raspberrypi.com/documentation/computers/configuration.html#wireless-networking-command-line).

For remote access, head to [Remote access](https://www.raspberrypi.com/documentation/computers/remote-access.html) and [SSH configuration](https://www.raspberrypi.com/documentation/computers/remote-access.html#ssh) documentation for setup instructions.

- In GUI, Go to Preferences → Raspberry Pi Configuration → Interfaces → Enable SSH, then `ssh pi@<pi-IP>`.
- In headless mode, create an `ssh` file and `userconf.txt` (See [Configuring a User](https://www.raspberrypi.com/documentation/computers/configuration.html#configuring-a-user)) on the boot partition of the SD card.

## Installation

You can follow the steps on the [official website](https://pi-hole.net/) to install Pi-hole on a Raspberry Pi 4 Model B.

On your Pi, clone the [Pi-hole repository](https://github.com/pi-hole/pi-hole) and run the installation script:

```bash
git clone --depth 1 https://github.com/pi-hole/pi-hole.git Pi-hole
cd "Pi-hole/automated install/"
sudo bash basic-install.sh
```

Follow through the installation wizard. [This article](https://privacyinternational.org/guide-step/4341/raspberry-pi-setup-and-run-pi-hole) has some good information about the wizard steps.

## Router DNS Configuration

Next, you need to configuring the router's DHCP clients to use Raspberry Pi as their DNS Server so that when the clients resolves IP address of an Ad server Pi-hole would block it. To do that you need a couple of things:

- Raspberry Pi's IP Address
- Access to router's web admin interface. You can find your router IP using `netstat -rn | grep default`. This returns the IP of the default gateway, for example `192.168.0.1`.

### Find Raspberry Pi's IP Address

There are [plenty of ways](https://www.raspberrypi.com/documentation/computers/remote-access.html#ip-address) to find your Raspberry Pi's IP address.

Since my Raspberry Pi supports mDNS by the [Avahi service](https://wiki.archlinux.org/title/avahi), `ping raspberrypi.local` and `ping6 raspberrypi.local` from my computer return Pi's IP addresses.

If that's not the case for you, you could try out the `nmap` method. Run `sudo nmap -sn <your-computer-IP>/24` to scan the devices on your local network and look for the hostname containing Raspberry Pi.

### Static IP Address

You'll want the Raspberry Pi's IP address to be static so that you won't need to reconfigure Pi-hole and your router if you restart Raspberry Pi.

To set static IP address on a Raspberry Pi, edit `/etc/dhcpcd.conf` and add the following. You can learn more about [dhcpcd here](https://wiki.archlinux.org/title/dhcpcd#Static_profile).

```bash
interface wlan0 # interface eth0 if you are using ethernet cable
static ip_address=<your-pi-IP>/24
static routers=<router-ip>
static domain_name_servers=
```

To apply the changes reboot using `sudo reboot` or bring down / bring up the network interface:

```bash
sudo ip link set wlan0 down
sudo ip link set wlan0 up
```




Also, reserve a static IP for the Raspberry Pi's MAC address in router admin page DHCP settings.

**Reconfigure Pi-hole**

If the IP Address of the Pi changes, you will need to reconfigure Pi-hole. Opening a terminal from Raspberry Pi and running `pihole -r` does the job.

### Configure Router DNS

Configuring DNS is pretty straightforward: Check the manual of your router brand, go to the router admin page and set DNS override to the IP address of the Raspberry Pi.

Note that it's [recommended](https://discourse.pi-hole.net/t/why-should-pi-hole-be-my-only-dns-server/3376) to use Pi-hole as the **single** DNS server (Or servers, if you care about redundancy) because your clients might send blocked requests to secondary DNS Servers.



## Pi-hole Admin

Go to `http://pi.hole/admin` or `http://<Your-Pi-IP>/admin` to manage Pi-hole.

To check Pi-hole service status, run `pihole status` on the command line on the Raspberry Pi.

You can also use dig to find out the performance of DNS queries. For example: `dig www.youtube.com | grep "Query time"` .

Posts