Eval | 谷粒 kuhung

工具评估进度

整体体验不是很好，特别图生图场景。兼容性没有想象中好。常出现权重找不到，或者说明不清楚的情况。

huggingface heroku

https://tabbyml.github.io/tabby/docs/self-hosting/docker/ 代码生成工具

docker run -it \
  --gpus all -p 8088:8080 -v $HOME/.tabby:/data \
  tabbyml/tabby \
  serve --model TabbyML/SantaCoder-1B --device cuda
  
# sudo apt-get install -y nvidia-container-toolkit
# sudo systemctl restart docker
# sudo docker restart chatgpt-next-web

docker run -itd \
  --gpus all -p 8080:8080 -v $HOME/data_n1p6/tabby/.tabby:/data \
  --name CodeLlama \
  --restart=always \
  tabbyml/tabby \
  serve --model TabbyML/CodeLlama-7B --device cuda
  
docker inspect --format='{{.Name}} {{.HostConfig.RestartPolicy.Name}}' $(docker ps -aq)
sudo lsof -i :8080 | awk 'NR!=1 {print $2}' | xargs sudo kill

sudo apt-get update
sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable"
apt-cache policy docker-ce
sudo apt install docker-ce