Crawl
Teach daily is web crawl by GoLang from web dev.to, freecodecamp.com, medium.com, hashnode.com, logrocket.com,infoq.com
Install / Use
/learn @chieund/CrawlREADME
Build Run Local
Change file app_example.yaml to app.yaml
cp app_example.yaml app.yaml
Build Docker
docker-compose up --build
Install package Golang
docker-compose exec crawl go mod tidy
Folder vendor
docker-compose exec crawl go mod vendor
Run Crawl
docker-compose exec crawl go run cmd/main.go
Use air autoload
docker-compose exec crawl air -c .air.conf
Deploy
Run file makefile build project into folder bin
make copy_template build_app_web build_app_crawl
Create Services in run in background (https://www.atpeaz.com/running-go-app-as-a-service-on-ubuntu/amp/)
Create Service and Run App Web
sudo nano /lib/systemd/system/app_web.service
Copy Content
[Unit]
Description=App Web
[Service]
Type=simple
Restart=always
RestartSec=5s
WorkingDirectory=/root/actions-runner/crawl/crawl/crawl/bin
ExecStart=/root/actions-runner/crawl/crawl/crawl/bin/app_web
[Install]
WantedBy=multi-user.target
sudo systemctl enable app_web
sudo systemctl start app_web
sudo systemctl status app_web
Run App Crawl
./app_crawl
Add CronTab
crontab -e
add cron time
*/60 * * * * /root/actions-runner/crawl/crawl/crawl/bin/app_crawl crawl-article
*/20 * * * * /root/actions-runner/crawl/crawl/crawl/bin/app_crawl crawl-article-detail
Reload cron run
sudo service cron reload
Website
http://techdaily.info/
Related Skills
node-connect
337.7kDiagnose OpenClaw node connection and pairing failures for Android, iOS, and macOS companion apps
xurl
337.7kA CLI tool for making authenticated requests to the X (Twitter) API. Use this skill when you need to post tweets, reply, quote, search, read posts, manage followers, send DMs, upload media, or interact with any X API v2 endpoint.
prose
337.7kOpenProse VM skill pack. Activate on any `prose` command, .prose files, or OpenProse mentions; orchestrates multi-agent workflows.
frontend-design
83.3kCreate distinctive, production-grade frontend interfaces with high design quality. Use this skill when the user asks to build web components, pages, or applications. Generates creative, polished code that avoids generic AI aesthetics.

