|
|
|
|
|
= 基于Nginx的高可用分布式网关开发手册
|
|
|
|
|
|
|
|
|
|
|
|
== 一. 开发环境搭建
|
|
|
|
|
|
=== 1. 开发环境组成
|
|
|
|
|
|
|
|
|
|
|
|
* 操作系统：Windows 11（需要支持WSL2，WSL2为Ubuntu-22.04）；
|
|
|
|
|
|
* 容器： Docker Desktop
|
|
|
|
|
|
* IDE： Visual Studio Code
|
|
|
|
|
|
* 编程语言：C语言为主
|
|
|
|
|
|
* 编译器： GCC
|
|
|
|
|
|
* 调试工具：gdb
|
|
|
|
|
|
* 其他工具：kimi.ai
|
|
|
|
|
|
|
|
|
|
|
|
=== 2. 开发环境安装
|
|
|
|
|
|
暂略
|
|
|
|
|
|
|
|
|
|
|
|
=== 3. 开发环境截图
|
|
|
|
|
|
|
|
|
|
|
|
* WSL2：不想装虚拟机软件，就用WSL2替代了
|
|
|
|
|
|
|
|
|
|
|
|
在windows系统查看wsl版本号：
|
|
|
|
|
|
|
|
|
|
|
|
image::./imgs/wsl.png[WSL版本]
|
|
|
|
|
|
进入wsl后，查看Linux系统版本号：
|
|
|
|
|
|
|
|
|
|
|
|
image::./imgs/wsl-linux.png[Linux版本号]
|
|
|
|
|
|
* Docker Desktop：通过创建多个docker容器模拟分布式计算环境：Nginx集群+后端服务器集群
|
|
|
|
|
|
|
|
|
|
|
|
Docker Desktop设置截图：
|
|
|
|
|
|
|
|
|
|
|
|
image::./imgs/DockerDesktop.png[Docker Desktop]
|
|
|
|
|
|
其中，Docker镜像仓库代理设置参考如下：
|
|
|
|
|
|
[source,json]
|
|
|
|
|
|
----
|
|
|
|
|
|
{
|
|
|
|
|
|
"builder": {
|
|
|
|
|
|
"gc": {
|
|
|
|
|
|
"defaultKeepStorage": "20GB",
|
|
|
|
|
|
"enabled": true
|
|
|
|
|
|
}
|
|
|
|
|
|
},
|
|
|
|
|
|
"experimental": false,
|
|
|
|
|
|
"registry-mirrors": [
|
|
|
|
|
|
"https://dockerproxy.com",
|
|
|
|
|
|
"https://hub-mirror.c.163.com",
|
|
|
|
|
|
"https://mirror.baidubce.com",
|
|
|
|
|
|
"https://ccr.ccs.tencentyun.com"
|
|
|
|
|
|
]
|
|
|
|
|
|
}
|
|
|
|
|
|
----
|
|
|
|
|
|
* Visual Studio Code，建议安装以下插件：插件安装方法见Visual Studio Code官网
|
|
|
|
|
|
1. C语言的插件群，安装方法见：link:https://code.visualstudio.com/docs/languages/cpp[C语言插件]
|
|
|
|
|
|
2. 安装WSL插件（可选）：远程连接WSL子系统，使用WSL子系统作为编译和运行环境，详见link:https://code.visualstudio.com/docs/remote/wsl-tutorial[在WSL进行远程开发]；
|
|
|
|
|
|
3. 安装Dev Container插件：远程连接Docker容器，以将Docker容器作为Nginx组件开发和调试环境，详见：link:https://code.visualstudio.com/docs/devcontainers/create-dev-container[创建Dev Container]
|
|
|
|
|
|
|
|
|
|
|
|
image::./imgs/dev-container.png[运行中的Dev Container]
|
|
|
|
|
|
|
|
|
|
|
|
image::./imgs/dev-container-2.png[Visual Studio Code连接Dev Container]
|
|
|
|
|
|
|
|
|
|
|
|
== 二. Nginx原理
|
|
|
|
|
|
=== 1. 下载Nginx源码
|
|
|
|
|
|
|
|
|
|
|
|
link:https://nginx.org/en/download.html[下载地址]
|
|
|
|
|
|
|
|
|
|
|
|
image::./imgs/download-nginx-src.png[下载Nginx源码]
|
|
|
|
|
|
|
|
|
|
|
|
=== 2. 编译Nginx源码
|
|
|
|
|
|
编译Nginx之前，需要提前安装GCC、Make、GDB等编译工具，安装过程略。
|
|
|
|
|
|
|
|
|
|
|
|
Nginx源码结构（不同下载路径，结构略有不同）
|
|
|
|
|
|
|
|
|
|
|
|
image::./imgs/nginx-src-structure.png[Nginx源码结构]
|
|
|
|
|
|
|
|
|
|
|
|
编译方法：link:https://nginx.org/en/docs/configure.html[编译Nginx源代码]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
=== 3. 一个开源实例模块
|
|
|
|
|
|
开源地址：link:https://github.com/yaoweibin/nginx_upstream_check_module[nginx_upstream_check_module]
|
|
|
|
|
|
|
|
|
|
|
|
=== 4. 编译开源组件
|
|
|
|
|
|
为了便于编译，我们将编译命令和参数写成shell脚本（`configure-health-check.sh`）：
|
|
|
|
|
|
[source,shell]
|
|
|
|
|
|
----
|
|
|
|
|
|
#!/bin/sh
|
|
|
|
|
|
|
|
|
|
|
|
# -g3 -gdwarf-2 允许在gdb调试时查看macro definitions以及macro expansion
|
|
|
|
|
|
# -O0 -DNDEBUG 允许compiler debugging
|
|
|
|
|
|
# --with-debug 开启nginx debugging
|
|
|
|
|
|
# -fexec-charset=GBK 允许nginx输出中文
|
|
|
|
|
|
./configure --with-cc-opt="-O0 -DNDEBUG -fexec-charset=GBK -g3 -gdwarf-2 -fsanitize=address -DNGX_DEBUG_PALLOC=1" --with-debug --prefix=/opt/nginx --with-http_ssl_module --add-module=/workspaces/cpp-5/nginx_upstream_check_module
|
|
|
|
|
|
----
|
|
|
|
|
|
其中，+++--prefix+++表示nginx安装目录；+++--with-debug+++允许nginx打开debug模式，并与+++-O0 -g3 -DNDEBUG -gdwarf-2+++配合，避免由于编译优化导致部分调试信息丢失；+++--add-module+++指定开源模块的源码路径。 +++-fsanitize=address -DNGX_DEBUG_PALLOC=1+++将屏蔽nginx memory pool，并允许内存debug
|
|
|
|
|
|
|
|
|
|
|
|
使用以下命令执行脚本：
|
|
|
|
|
|
[source,shell]
|
|
|
|
|
|
----
|
|
|
|
|
|
sh configure-health-check.sh
|
|
|
|
|
|
----
|
|
|
|
|
|
|
|
|
|
|
|
然后再分别执行make和make install安装nginx：
|
|
|
|
|
|
[source,shell]
|
|
|
|
|
|
----
|
|
|
|
|
|
make
|
|
|
|
|
|
make install
|
|
|
|
|
|
----
|
|
|
|
|
|
清理编译结果：
|
|
|
|
|
|
[source,shell]
|
|
|
|
|
|
----
|
|
|
|
|
|
make clean
|
|
|
|
|
|
----
|
|
|
|
|
|
|
|
|
|
|
|
=== 5. 调试Nginx
|
|
|
|
|
|
执行以下命令，进入gdb调试界面
|
|
|
|
|
|
[source,shell]
|
|
|
|
|
|
----
|
|
|
|
|
|
gdb /opt/nginx/sbin/nginx
|
|
|
|
|
|
----
|
|
|
|
|
|
image::./imgs/gdb-entry.png[打开gdb调试]
|
|
|
|
|
|
在gdb命令行中开启多线程调试：当主进程fork出子进程后，将自动对子进程调试
|
|
|
|
|
|
[source,shell]
|
|
|
|
|
|
----
|
|
|
|
|
|
set follow-fork-mode child
|
|
|
|
|
|
set detach-on-fork on
|
|
|
|
|
|
----
|
|
|
|
|
|
image::./imgs/gdb-mutil-threads.png[多线程调试开关]
|
|
|
|
|
|
在想要调试的地方设置断点后，开始调试：
|
|
|
|
|
|
[source,shell]
|
|
|
|
|
|
----
|
|
|
|
|
|
br ngx_http_upstream_check_status_handler
|
|
|
|
|
|
...
|
|
|
|
|
|
start
|
|
|
|
|
|
...
|
|
|
|
|
|
cont
|
|
|
|
|
|
....
|
|
|
|
|
|
----
|
|
|
|
|
|
image::./imgs/debug-nginx.png[调试nginx]
|
|
|
|
|
|
|
|
|
|
|
|
关于gdb调试指令可参考：link:https://wizardforcel.gitbooks.io/100-gdb-tips/content/index.html[gdb调试小技巧]
|
|
|
|
|
|
|
|
|
|
|
|
== 三. 网关原理
|
|
|
|
|
|
|
|
|
|
|
|
参考：
|
|
|
|
|
|
|
|
|
|
|
|
1. 负载均衡解释：link:https://www.nginx.org.cn/article/detail/440[]
|
|
|
|
|
|
2. 高可用配置示例：link:https://blog.csdn.net/IT_10/article/details/89365436[]
|
|
|
|
|
|
3. nginx开发参考1：link:https://tengine.taobao.org/book/[]
|
|
|
|
|
|
4. nginx开发参考2：link:https://www.nginx.org.cn/article/detail/443[]
|
|
|
|
|
|
5. nginx中文配置手册：link:https://wizardforcel.gitbooks.io/nginx-doc/content/index.html[]
|
|
|
|
|
|
6. nginx开发参考3：link:https://www.kancloud.cn/kancloud/master-nginx-develop/51798[]
|
|
|
|
|
|
|
|
|
|
|
|
== 四. 开发流程
|
|
|
|
|
|
|
|
|
|
|
|
== 五. 其他
|
|
|
|
|
|
|
|
|
|
|
|
== 六. 参考资料
|
