Rocky Linux上Harbor高可用集群部署指南

1. 项目概述在现代化IT基础设施中容器镜像仓库的高可用部署已成为企业级DevOps实践的标配。Rocky Linux作为RHEL的完美替代品结合Harbor的企业级镜像管理能力能够构建出稳定可靠的容器化基础设施。本文将详细记录一个生产级Harbor高可用集群在Rocky Linux上的完整部署过程。2. 环境规划与准备2.1 硬件资源配置建议对于生产环境建议采用以下配置控制节点3台每台8核CPU/16GB内存/100GB存储工作节点2台每台16核CPU/32GB内存/200GB存储负载均衡器2台每台4核CPU/8GB内存/50GB存储注意实际配置应根据预期负载调整镜像存储量大的场景需要额外规划存储节点2.2 软件版本选型经过实际测试验证的稳定版本组合操作系统Rocky Linux 8.6Harborv2.6.2Docker20.10.17Docker Composev2.6.0PostgreSQL13.5Redis6.2.63. 基础环境部署3.1 Rocky Linux系统初始化在所有节点执行以下初始化操作# 设置主机名 hostnamectl set-hostname node1 # 关闭SELinux sed -i s/SELINUXenforcing/SELINUXdisabled/g /etc/selinux/config setenforce 0 # 配置防火墙 firewall-cmd --permanent --add-port80/tcp firewall-cmd --permanent --add-port443/tcp firewall-cmd --permanent --add-port4443/tcp firewall-cmd --reload # 安装基础工具 dnf install -y vim wget curl net-tools lsof3.2 Docker环境部署# 安装依赖 dnf install -y yum-utils device-mapper-persistent-data lvm2 # 添加Docker仓库 yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo # 安装Docker dnf install -y docker-ce docker-ce-cli containerd.io # 配置Docker mkdir -p /etc/docker cat /etc/docker/daemon.json EOF { exec-opts: [native.cgroupdriversystemd], log-driver: json-file, log-opts: { max-size: 100m }, storage-driver: overlay2, registry-mirrors: [https://registry.docker-cn.com] } EOF # 启动服务 systemctl enable docker systemctl start docker4. Harbor高可用集群部署4.1 负载均衡配置使用Nginx作为Harbor前端负载均衡器upstream harbor { server 192.168.1.101:80; server 192.168.1.102:80; server 192.168.1.103:80; } server { listen 80; server_name harbor.example.com; location / { proxy_pass http://harbor; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } }4.2 数据库集群部署PostgreSQL主从配置# 主节点配置 vim /var/lib/pgsql/13/data/postgresql.conf # 修改以下参数 wal_level replica max_wal_senders 3 wal_keep_size 1GB # 从节点配置 pg_basebackup -h 192.168.1.101 -U replicator -D /var/lib/pgsql/13/data -P echo standby_mode on /var/lib/pgsql/13/data/recovery.conf echo primary_conninfo host192.168.1.101 port5432 userreplicator passwordyourpassword /var/lib/pgsql/13/data/recovery.conf4.3 Redis哨兵模式配置三节点Redis哨兵配置示例# redis.conf bind 0.0.0.0 protected-mode no port 6379 daemonize yes requirepass yourpassword # sentinel.conf sentinel monitor mymaster 192.168.1.101 6379 2 sentinel down-after-milliseconds mymaster 5000 sentinel failover-timeout mymaster 10000 sentinel auth-pass mymaster yourpassword4.4 Harbor安装与配置下载并解压Harbor安装包wget https://github.com/goharbor/harbor/releases/download/v2.6.2/harbor-offline-installer-v2.6.2.tgz tar xvf harbor-offline-installer-v2.6.2.tgz cd harbor修改harbor.yml配置文件hostname: harbor.example.com http: port: 80 https: port: 443 certificate: /data/cert/server.crt private_key: /data/cert/server.key harbor_admin_password: Harbor12345 database: password: root123 max_idle_conns: 100 max_open_conns: 900 data_volume: /data clair: updaters_interval: 12 jobservice: max_job_workers: 10 notification: webhook_job_max_retry: 3 chart: absolute_url: disabled log: level: info local: rotate_count: 50 rotate_size: 200M location: /var/log/harbor执行安装脚本./install.sh --with-clair --with-chartmuseum5. 存储方案集成5.1 MinIO集群部署作为Harbor的存储后端MinIO提供高可用对象存储# 下载MinIO wget https://dl.min.io/server/minio/release/linux-amd64/minio chmod x minio mv minio /usr/local/bin/ # 创建启动脚本 cat /etc/systemd/system/minio.service EOF [Unit] DescriptionMinIO Afternetwork.target [Service] Typesimple Userminio Groupminio ExecStart/usr/local/bin/minio server http://node{1...4}/data{1...4} \ --console-address :9001 Restartalways [Install] WantedBymulti-user.target EOF # 启动服务 systemctl daemon-reload systemctl enable minio systemctl start minio5.2 Harbor配置MinIO存储修改harbor.ymlstorage_service: s3: accesskey: yourminioaccesskey secretkey: yourminiosecretkey region: us-east-1 bucket: harbor endpoint: https://minio.example.com secure: true chunksize: 5242880 rootdirectory: /harbor6. 运维与监控6.1 日常维护命令# 查看服务状态 docker-compose ps # 停止Harbor docker-compose down -v # 启动Harbor docker-compose up -d # 备份数据 tar czvf harbor-data-$(date %Y%m%d).tar.gz /data/database /data/registry6.2 Prometheus监控配置Harbor监控指标采集配置示例scrape_configs: - job_name: harbor static_configs: - targets: [harbor.example.com] metrics_path: /metrics scheme: https tls_config: insecure_skip_verify: true7. 常见问题排查7.1 镜像推送失败可能原因及解决方案证书问题确保客户端配置了正确的CA证书存储空间不足检查MinIO存储配额网络问题验证节点间网络连通性7.2 登录认证失败检查步骤确认PostgreSQL服务正常运行验证Redis缓存服务状态检查Harbor核心服务日志docker logs harbor-core7.3 高可用切换异常故障转移测试方法手动停止主节点Harbor服务观察负载均衡器自动切换验证从节点服务接管情况8. 性能优化建议数据库优化增加PostgreSQL连接池大小配置合适的WAL日志保留策略存储优化使用SSD存储后端启用MinIO纠删码功能网络优化配置节点间万兆网络启用TCP BBR拥塞控制算法# 启用BBR echo net.core.default_qdiscfq /etc/sysctl.conf echo net.ipv4.tcp_congestion_controlbbr /etc/sysctl.conf sysctl -p9. 安全加固措施证书管理使用Lets Encrypt自动续期证书配置证书轮换策略访问控制启用Harbor项目配额配置细粒度权限策略审计日志集中收集Harbor操作日志配置日志告警规则# 日志审计配置示例 cat /etc/rsyslog.d/harbor.conf EOF module(loadimfile PollingInterval10) input(typeimfile File/var/log/harbor/*.log Tagharbor Severityinfo Facilitylocal7) EOF10. 扩展功能集成10.1 与CI/CD系统对接Jenkins集成示例pipeline { agent any environment { HARBOR_CREDS credentials(harbor-credential) } stages { stage(Build) { steps { sh docker build -t harbor.example.com/myproject/app:${BUILD_NUMBER} . } } stage(Push) { steps { sh docker login -u ${HARBOR_CREDS_USR} -p ${HARBOR_CREDS_PSW} harbor.example.com docker push harbor.example.com/myproject/app:${BUILD_NUMBER} } } } }10.2 镜像扫描策略配置启用Clair漏洞扫描clair: updaters_interval: 12 http_proxy: http://proxy.example.com:3128 https_proxy: http://proxy.example.com:3128 no_proxy: 127.0.0.1,localhost,.example.com11. 升级与迁移11.1 Harbor版本升级步骤备份数据库和配置文件下载新版安装包修改配置适配新版本执行升级脚本./prepare --with-clair --with-chartmuseum docker-compose up -d11.2 数据迁移方案跨集群迁移流程使用harbor-migrator工具导出元数据同步镜像存储数据在新集群导入元数据验证数据完整性# 导出数据 harbor-migrator export --config harbor.yml --output backup.tar.gz # 导入数据 harbor-migrator import --config harbor.yml --input backup.tar.gz12. 灾备恢复方案12.1 全量备份策略建议备份以下内容Harbor数据库dump配置文件目录证书文件镜像存储数据# 创建每日备份任务 0 2 * * * /usr/bin/pg_dump -U postgres -d registry /backup/harbor-db-$(date \%Y\%m\%d).sql 0 3 * * * tar czvf /backup/harbor-config-$(date \%Y\%m\%d).tar.gz /harbor/config12.2 灾难恢复流程恢复步骤在新环境部署基础服务恢复数据库备份还原配置文件同步存储数据启动Harbor服务13. 最佳实践总结经过多个生产环境部署经验总结以下关键点容量规划要预留30%以上余量定期测试高可用切换功能监控关键指标存储使用率、API响应时间、认证成功率建立完善的备份验证机制实际部署中发现配置合理的PostgreSQL连接池参数和Redis缓存策略可以显著提升Harbor在高并发场景下的性能表现。建议根据实际负载情况持续优化这些关键参数。