林占宇

UE5 中编译使用 Protobuf

Thu, 23 Oct 2025 00:00:00 +0000

UE 中第三方库的编译

编译主要得有 -fPIC 参数, 生编译器产生位置无关代码(Positio Independent Code)

产生的代码没有绝对地址, 全部使用相对地址, 故代码可以被加载到内存的任意位置, 都可以正确执行.

Windows 编译

Linux 编译

编译器的选择:

Linux下的编译器有GCC和clang可以选择, 由于UE使用的是自带的 clang 编译器, 所以我们也在系统中安装 clang 吧!

直接编译放入UE工程后, 编译链接时会报错如下:

[25/114] Link (lld) libUnrealEditor-MineNet.so                                                                                                                                                                                                                                               [453/1851]
ld.lld: error: undefined symbol: std::allocator<char>::allocator()                                                                                                                                                                                                                                     
>>> referenced by arenastring.cc                                                                                                                                                                                                                                                                       
>>>               arenastring.cc.o:(google::protobuf::internal::LazyString::Init[abi:cxx11]() const) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a                                                                                     
>>> referenced by arenastring.cc                                                                                                                                                                                                                                                                       
>>>               arenastring.cc.o:(google::protobuf::internal::(anonymous namespace)::CreateString(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf
.a                                                                                                                                                                                                                                                                                                     
>>> referenced by arenastring.cc                                                                                                                   
>>>               arenastring.cc.o:(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>* google::protobuf::Arena::Create<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, char const*, unsigned long>(google::protobuf::Arena*, char const*&&
, unsigned long&&)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a                                                                                                                                                                      
>>> referenced 180 more times                                                                                                                                                                                                                                                                          
>>> did you mean: std::allocator<int>::allocator()                                                                                                                                                                                                                                                     
>>> defined in: /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a(descriptor.cc.o)                                                                                                                                                                    
ld.lld: error: undefined symbol: std::string::empty() const                                                                                                                                                                                                                                            
>>> referenced by miniodata.pb.cc                                                                                                                                                                                                                                                                      
>>>               miniodata.pb.cc.o:(FNetMSG::FNetMSG(FNetMSG const&)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protocol/lib/libProtocol.a
>>> referenced by miniodata.pb.cc
>>>               miniodata.pb.cc.o:(FNetMSG::_InternalSerialize(unsigned char*, google::protobuf::io::EpsCopyOutputStream*) const) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protocol/lib/libProtocol.a
>>> referenced by miniodata.pb.cc
>>>               miniodata.pb.cc.o:(FNetMSG::ByteSizeLong() const) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protocol/lib/libProtocol.a
>>> referenced 73 more times
ld.lld: error: undefined symbol: std::ios_base::Init::Init()
>>> referenced by arena.cc
>>>               arena.cc.o:(__static_initialization_and_destruction_0(int, int)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a
>>> referenced by generated_message_util.cc
>>>               generated_message_util.cc.o:(__static_initialization_and_destruction_0(int, int)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a
>>> referenced by repeated_field.cc
>>>               repeated_field.cc.o:(__static_initialization_and_destruction_0(int, int)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a
>>> referenced 35 more times
ld.lld: error: undefined symbol: std::ios_base::Init::~Init()
>>> referenced by arena.cc
>>>               arena.cc.o:(__static_initialization_and_destruction_0(int, int)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a
>>> referenced by generated_message_util.cc
>>>               generated_message_util.cc.o:(__static_initialization_and_destruction_0(int, int)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a
>>> referenced by repeated_field.cc
>>>               repeated_field.cc.o:(__static_initialization_and_destruction_0(int, int)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a
>>> referenced 35 more times
ld.lld: error: undefined symbol: google::protobuf::internal::ArenaStringPtr::Set(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, google::protobuf::Arena*)
>>> referenced by arenastring.h:409 (/mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/include/google/protobuf/arenastring.h:409)
>>>               /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Intermediate/Build/Linux/x64/UnrealEditor/Development/MineNet/NetworkMngr.cpp.o:(UNetworkMngr::SendMSG(MineUIData&, TDelegate<void (MineUIData const&), FDefaultDelegateUserPolicy> const&))
>>> referenced by arenastring.h:409 (/mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/include/google/protobuf/arenastring.h:409)
>>>               /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Intermediate/Build/Linux/x64/UnrealEditor/Development/MineNet/NetworkMngr.cpp.o:(UNetworkMngr::SendRequest(FString const&, TDelegate<void (bool, MineInfo const&), FDefaultDelegateUserPolicy> const&))
>>> referenced by arenastring.h:409 (/mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/include/google/protobuf/arenastring.h:409)
>>>               /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Intermediate/Build/Linux/x64/UnrealEditor/Development/MineNet/NetworkMngr.cpp.o:(UNetworkMngr::SendRequest(FString const&, TDelegate<void (bool, FString const&), FDefaultDelegateUserPolicy> const&))
>>> referenced 5 more times
ld.lld: error: undefined symbol: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>::~basic_string()
>>> referenced by generated_message_util.cc
>>>               generated_message_util.cc.o:(google::protobuf::internal::DestroyString(void const*)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a
>>> referenced by extension_set.cc
>>>               extension_set.cc.o:(google::protobuf::internal::(anonymous namespace)::Register(google::protobuf::internal::ExtensionInfo const&)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a
>>> referenced by extension_set.cc
>>>               extension_set.cc.o:(google::protobuf::internal::(anonymous namespace)::Register(google::protobuf::internal::ExtensionInfo const&)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a
>>> referenced 1497 more times
ld.lld: error: undefined symbol: google::protobuf::internal::ArenaStringPtr::Set(std::string const&, google::protobuf::Arena*)
>>> referenced by miniodata.pb.cc
>>>               miniodata.pb.cc.o:(FNetMSG::FNetMSG(FNetMSG const&)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protocol/lib/libProtocol.a
>>> referenced by miniodata.pb.cc
>>>               miniodata.pb.cc.o:(FNetMSG::MergeImpl(google::protobuf::Message&, google::protobuf::Message const&)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protocol/lib/libProtocol.a
>>> referenced by miniodata.pb.cc
>>>               miniodata.pb.cc.o:(FNetCMD::FNetCMD(FNetCMD const&)) in archive /mnt/disk/jenkins/workspace/Dev-MineMind-Linux/Source/ThirdParty/Protocol/lib/libProtocol.a
>>> referenced 35 more times

如何来分析这个问题?

首先要观察 libprotobuf.a 和引用该静态库的 MineNet/XXX.o

# 
nm -C Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a | grep "basic_string" | head
# 得到结果表示使用 libc++ 编译的
nm -C Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a | c++filt | grep "std::__1" | head
# 得到结果表示使用 libstdc++ 编译的
nm -C Source/ThirdParty/Protobuf/3.21.7/lib/libprotobuf.a | c++filt | grep "std::__cxx11" | head
# 检查 UE5 编译后用的是哪种方式
nm -C Intermediate/Build/Linux/x64/MineMindClient/Development/MineNet/MineNet.cpp.o | c++filt | grep "std::__1" | head

通过以上命令我确定了我自编的UE5是使用了 clang + libc++ 的方式.

所以只需要让 Protobuf 编译的和 UE 的方式相同了即可.

sudo apt install clang
# 使用 libc++ 
sudo apt install libc++-dev libc++abi-dev

export CC=clang
export CXX=clang++

cmake -B build \
 -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
 -Dprotobuf_BUILD_TESTS=OFF \
 -DCMAKE_CXX_FLAGS="-stdlib=libc++ -fPIC -DGOOGLE_PROTOBUF_INTERNAL_DONATE_STEAL_INLINE=1" \
 -DCMAKE_CXX_COMPILER=clang++ \
 -DCMAKE_C_COMPILER=clang \
 -DCMAKE_INSTALL_PREFIX=<YourWorkspace>/Source/ThirdParty/Protobuf/3.21.7

cmake --build build --config Release --target install -j8

Ubuntu22.04 设置无法打开

Tue, 20 May 2025 00:00:00 +0000

记录一次Ubuntu下设置打不开的奇怪问题

$ echo $DISPLAY
echo $XDG_SESSION_TYPE
gnome-control-center --verbose
:0
wayland
总线错误 (核心已转储)

尝试重新安装 gnome-control-center, gnome-desktop 之类均未能修复, 只好挂上GDB看看了

$ gdb gnome-control-center
GNU gdb (Ubuntu 12.1-0ubuntu1~22.04.2) 12.1
Copyright (C) 2022 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Registered pretty printers for UE classes
Registered pretty printers for UE classes
Registered pretty printers for UE classes
Reading symbols from gnome-control-center...
(No debugging symbols found in gnome-control-center)
(gdb) r
Starting program: /usr/bin/gnome-control-center 

Program received signal SIGBUS, Bus error.
memset () at ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:283
283	../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: 没有那个文件或目录.
(gdb) l
278	in ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S
(gdb) c
Continuing.

Program terminated with signal SIGBUS, Bus error.
The program no longer exists.

那么我们发现 memset 这个函数 Crash 了. 我们检查更底层的gnome依赖的文件是否出现了损坏或异常.

sudo apt update
sudo apt install debsums
sudo debsums -s

图形驱动和渲染相关库损坏：

$ sudo debsums -s
debsums: changed file /usr/lib/firefox/libxul.so (from firefox package)
debsums: changed file /usr/lib/x86_64-linux-gnu/dri/kms_swrast_dri.so (from libgl1-mesa-dri:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/dri/r600_dri.so (from libgl1-mesa-dri:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/dri/radeonsi_dri.so (from libgl1-mesa-dri:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/dri/swrast_dri.so (from libgl1-mesa-dri:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/dri/virtio_gpu_dri.so (from libgl1-mesa-dri:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/dri/vmwgfx_dri.so (from libgl1-mesa-dri:amd64 package)
debsums: changed file /usr/lib/i386-linux-gnu/dri/kms_swrast_dri.so (from libgl1-mesa-dri:i386 package)
debsums: changed file /usr/lib/i386-linux-gnu/dri/r600_dri.so (from libgl1-mesa-dri:i386 package)
debsums: changed file /usr/lib/i386-linux-gnu/dri/radeonsi_dri.so (from libgl1-mesa-dri:i386 package)
debsums: changed file /usr/lib/i386-linux-gnu/dri/swrast_dri.so (from libgl1-mesa-dri:i386 package)
debsums: changed file /usr/lib/i386-linux-gnu/dri/virtio_gpu_dri.so (from libgl1-mesa-dri:i386 package)
debsums: changed file /usr/lib/i386-linux-gnu/dri/vmwgfx_dri.so (from libgl1-mesa-dri:i386 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/libLLVM-13.so.1 (from libllvm13:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/libLLVM-14.so.1 (from libllvm14:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/libLLVM-15.so.1 (from libllvm15:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/libQt5WebEngineCore.so.5.15.9 (from libqt5webenginecore5:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/libwebkit2gtk-4.0.so.37.72.6 (from libwebkit2gtk-4.0-37:amd64 package)
debsums: changed file /lib/systemd/system/logrotate.timer (from logrotate package)
debsums: changed file /usr/lib/x86_64-linux-gnu/dri/r600_drv_video.so (from mesa-va-drivers:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/dri/radeonsi_drv_video.so (from mesa-va-drivers:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/dri/virtio_gpu_drv_video.so (from mesa-va-drivers:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/vdpau/libvdpau_r600.so.1.0.0 (from mesa-vdpau-drivers:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/vdpau/libvdpau_radeonsi.so.1.0.0 (from mesa-vdpau-drivers:amd64 package)
debsums: changed file /usr/lib/x86_64-linux-gnu/vdpau/libvdpau_virtio_gpu.so.1.0.0 (from mesa-vdpau-drivers:amd64 package)
debsums: changed file /usr/share/misc/pci.ids (from pci.ids package)
debsums: changed file /usr/lib/thunderbird/libxul.so (from thunderbird package)
debsums: changed file /usr/lib/thunderbird/omni.ja (from thunderbird package)

重装受损核心库

sudo apt install --reinstall \
libgl1-mesa-dri:amd64 \
libgl1-mesa-dri:i386 \
mesa-va-drivers:amd64 \
mesa-vdpau-drivers:amd64 \
libllvm13:amd64 \
libllvm14:amd64 \
libllvm15:amd64 \
libwebkit2gtk-4.0-37:amd64 \
libqt5webenginecore5:amd64 \
logrotate \
firefox \
thunderbird \
pci.ids

再次尝试

$ gdb gnome-control-center
run

成功了!

gitea 的烦人认证

Wed, 05 Jun 2024 00:00:00 +0000

gitea 默认启用了SSH的RSA密钥使用3072的长度

在 custom/conf/app.ini 中添加下面的内容使其能使用正常的 RAS 密钥

[ssh.minimum_key_sizes]
RSA = 2047

git 检查哪里出现了问题

GIT_SSH_COMMAND=”ssh -v “ git clone you-repo-addr

vscode 不能代码跳转

Tue, 19 Dec 2023 00:00:00 +0000

VSCode无法代码跳转的修复

最近 VSCode 中Python代码无法跳转了,恰巧近期还修改了一些工作区配置,这下还以为是自己改错了什么配置各种回退折腾.无果.

今天突然想起vscode的这个自动升级是不是有问题.把这个1.85.1版本的vscode回退到1.79.2后果然代码跳转恢复好用了.

Armbian 上编译 synergy-core

Sun, 05 Nov 2023 00:00:00 +0000

sudo apt -y install qtcreator qtbase5-dev cmake make g++ xorg-dev libssl-dev libx11-dev libsodium-dev libgl1-mesa-glx libegl1-mesa libcurl4-openssl-dev libavahi-compat-libdnssd-dev qtdeclarative5-dev libqt5svg5-dev libsystemd-dev
sudo apt install -y qttools5-dev libnotify-dev libgdk-pixbuf2.0-dev libglib2.0-dev 

# cmake 常规套路
cd build
cmake ..
make
sudo make install

数学统计的实际意义

Wed, 12 Apr 2023 00:00:00 +0000

标准差

标准差是指一组数据的离散程度或变化量的一种度量，通常用符号 σ 表示。标准差越大，说明数据的离散程度越高，反之则说明数据越接近平均值。标准差的计算公式如下：

\[\sigma=\sqrt{\frac{1}{N}\sum_{i=1}^{N}(x_i-\mu)^2}\]

其中，$\sigma$ 表示标准差，$x_i$ 表示第 $i$ 个数据点，$\mu$ 表示数据的均值，$n$ 表示数据点的数量。

在实际应用中，标准差经常用于衡量数据的稳定性和风险程度，例如在金融领域中，投资者经常使用标准差来评估股票或基金的风险水平。标准差还可以用于判断数据是否符合正态分布。如果一组数据符合正态分布，那么其约 68% 的数据会落在平均值的一个标准差内，约 95% 的数据会落在平均值的两个标准差内，约 99.7% 的数据会落在平均值的三个标准差内。

残差

在统计学中，残差指的是一个实际观测值与对应的模型预测值之间的差异。也就是说，它是实际值与预测值之间的偏差，表示了模型未能解释的部分。如果一个模型能够完全解释实际观测值的变异，那么其残差将为零。

在机器学习中，残差可以用来评估模型的性能和预测精度。更准确地说，残差是在训练模型时用来计算误差的。模型的目标是最小化残差的平方和，这个值也被称为损失函数。最小化损失函数的过程就是模型的训练过程，即找到最优的模型参数，使得预测值和实际值之间的差异最小化。

在回归分析中，残差可以用来检验回归模型的适当性和是否符合假设。如果残差的方差不均匀或存在自相关性，那么就可能需要对模型进行调整。

用Cython来加速numpy数据操作

Fri, 12 Aug 2022 00:00:00 +0000

Cython是可以方便地用类Python语法代码编译生成C代码,免除多平台下复杂的C++工程配置.

Python中往往使用numpy来处理大量数值,对其计算时避免不了使用循环.循环数值计算再合并GIL造成的严重的CPU性能损耗.

那么把 numpy 数据对象转为 C 数组然后执行无GIL的本地编译计算.

Step 1 . np.array convert to C array

def const c_type[:] view = ndarray

这个视图操作会产生 memroy copy.

ndarray.data ndarray.ctypes.data

Step 2 . C ptr -> C array

很简单 &view[0] 即可得到地址

Step 3 . numerical calculation

val = ptr[0] + ptr[1]

Step 4 . 测试

必须要有完备的单元测试对数值结果进行对比测试

不得不提的一些问题

1. 数据类型问题

目前还没有找到 numpy data buffer 直接调用对应 C++ template 函数的方法

Linux 开发中使用VSCode连接进程PID调试

Sun, 07 Nov 2021 00:00:00 +0000

VScode连接 Python 进程调试时出现 “timed out waiting for debug server to connect” 提示, 连接失败.

设置为许可模式

修改 /etc/sysctl.d/10-ptrace.conf 文件中的 ptrace 的值

kernel.yama.ptrace_scope = 0

重新加载该文件

sysctl -p /etc/sysctl.d/10-ptrace.conf

这样操作后 F5 启动调试即可通过PID连接相应进程进行调试了.

Linux 开发中查看日志

Wed, 20 Oct 2021 00:00:00 +0000

多日志文件查看

tail -f log/*.log

Ubuntu 18.04 中安装 VIM8

Thu, 30 Sep 2021 00:00:00 +0000

VIM 8.2 释出已经有一段时间了. Ubuntu18.04自带的VIM7已经无法和其它平台同步插件配置了. 那么就把Ubuntu系统中的VIM升级一下吧.

安装

sudo add-apt-repository ppa:jonathonf/vim
sudo apt install vim
# sudo apt upgrade -y

移除

sudo add-apt-repository ppa:jonathonf/vim
sudo apt upgrade -y