本地部署IndexTTS-2-Demo
严振杰

IndexTTS2 是 B 站于 2025 年 9 月开源的,9 月份一直想玩一下,但是 9 月工作太忙了没有尝试,国庆期间趁热打铁来本地部署玩一下。

IndexTTS2 是首个自回归零样本 TTS 系统,支持精准时长控制、音色情感解耦、多模态输入等创新功能,可通过官网获取源码及技术文档。

特色:

  • 音色克隆:仅需 10 秒参考音频即可精准还原音色、口音及说话节奏。
  • 音色与情感解耦:支持独立指定音色来源和情绪来源,实现音色与情感的分离控制。
  • 情绪调节:在音色与情感解耦基础上,通过参考音频或文本描述调整愤怒、喜悦等情绪权重。
  • 多模态输入:支持文本描述、音频提示或情感向量控制语调,覆盖音色还原与情绪调节的输入方式。
  • 影视级应用:音画同步误差<0.02%,适配视频配音场景。
  • 实践案例:已应用于《让子弹飞》《甄嬛传》等经典影视作品 AI 配音。

1、项目获取

有 2 种获取方式。

  1. 如果你可以访问 huggingface,那么:

  2. 如果不能访问 huggingface,那么访问国内的 modelscope:

以上 2 个 Demo 地址都是可以在线体验的。

2、本地安装

本地部署非常简单,但是中间会遇到 openfst 和 pynini 的一些编译问题。

我的环境:

  • MacOS Sequia 15.7
  • Command_Line_Tools_for_Xcode_16.4_MacOS_15.3
  • Homebrew 4.6.15
  • pyenv 2.6.8
  • Python 3.10.12
1
2
3
4
5
6
7
$ brew list
==> Formulae
autoconf gettext libunistring m4 openfst pkgconf readline wget zlib
bzip2 libidn2 llvm mpdecimal [email protected] pyenv sqlite xz zstd
ca-certificates libtommath lz4 ncurses openssl@3 [email protected] tcl-tk z3

==> Casks

把项目拉到本地后,激活 python 环境,安装依赖:

1
pip3 install -r requirements.txt

当然,肯定是失败了,很顺利的话就不会有这篇文章了,让我们来看看发生了什么:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
...
copying pynini/export/py.typed -> build/lib.macosx-15.7-arm64-cpython-39/pynini/export
running build_ext
building '_pywrapfst' extension
creating build/temp.macosx-15.7-arm64-cpython-39/extensions
clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -DOPENSSL_NO_SSL3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/Users/Harry/Workspace/ModelScope/IndexTTS-2-Demo/.venv/include -I/Users/Harry/.pyenv/versions/3.9.23/include/python3.9 -c extensions/_pywrapfst.cpp -o build/temp.macosx-15.7-arm64-cpython-39/extensions/_pywrapfst.o -std=c++17 -Wno-register -Wno-deprecated-declarations -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char -stdlib=libc++ -mmacosx-version-min=10.12
extensions/_pywrapfst.cpp:1191:10: fatal error: 'fst/util.h' file not found
1191 | #include <fst/util.h>
| ^~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang++' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pynini
Failed to build pynini
ERROR: Could not build wheels for pynini, which is required to install pyproject.toml-based projects

核心是'fst/util.h' file not found,应该不存在 openfst,那么安装brew install openfst,然后再次pip install -r requirements.txt,依旧失败:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
building '_pywrapfst' extension
creating build/temp.macosx-15.7-arm64-cpython-39/extensions
clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -DOPENSSL_NO_SSL3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/Users/Harry/Workspace/ModelScope/IndexTTS-2-Demo/.venv/include -I/Users/Harry/.pyenv/versions/3.9.23/include/python3.9 -c extensions/_pywrapfst.cpp -o build/temp.macosx-15.7-arm64-cpython-39/extensions/_pywrapfst.o -std=c++17 -Wno-register -Wno-deprecated-declarations -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char -stdlib=libc++ -mmacosx-version-min=10.12
clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -DOPENSSL_NO_SSL3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -bundle -undefined dynamic_lookup build/temp.macosx-15.7-arm64-cpython-39/extensions/_pywrapfst.o -L/Users/Harry/.pyenv/versions/3.9.23/lib -lfstfarscript -lfstfar -lfstscript -lfst -lm -ldl -o build/lib.macosx-15.7-arm64-cpython-39/_pywrapfst.cpython-39-darwin.so
ld: library 'fstfarscript' not found
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/usr/bin/clang++' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pynini
Failed to build pynini
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> pynini

看着是 pynini 编译安装失败了,然后还注意到library 'fstfarscript' not found,网上查资料看是 openfst 和 pynini 有不匹配,其实从日志可以看出来不是缺少库,而是库里面缺少fstfarscript,有答案推荐安装 openfst 的开发版:

1
2
3
4
5
6
7
8
# 安装openfst开发版
brew reinstall openfst --build-from-source

# 设置连接器路径
export LDFLAGS="-L$(brew --prefix openfst)/lib $LDFLAGS"

# 手动尝试安装pynini
pip install pynini --no-cache-dir --global-option=build_ext --global-option="-L$(brew --prefix openfst)/lib"

OK,这里 pynini 安装成功了:

1
2
3
4
5
6
7
8
9
10
11
12
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/, https://pypi.tuna.tsinghua.edu.cn/simple/, https://mirrors.ustc.edu.cn/pypi/simple/, https://pypi.org/simple/
Processing /Users/Harry/Workspace/ModelScope/IndexTTS-2-Demo/pynini-2.1.7
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: pynini
Building wheel for pynini (pyproject.toml) ... done
Created wheel for pynini: filename=pynini-2.1.7-cp310-cp310-macosx_15_0_arm64.whl size=2023561 sha256=e2a86e63d5dfe7de6ef277e56d6d1f08016003410948b21259dc05b2c72e8b69
Stored in directory: /Users/Harry/Library/Caches/pip/wheels/81/07/cb/9d3f76d8da6b8b6791674ef8dc617e86bba48058c1165d11bb
Successfully built pynini
Installing collected packages: pynini
Successfully installed pynini-2.1.7

我很高兴的继续安装依赖pip install -r requirements.txt,实时证明我高兴的太早了:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
...
building '_pynini' extension
clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -DOPENSSL_NO_SSL3 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -Iextensions -I/Users/Harry/Workspace/ModelScope/IndexTTS-2-Demo/.venv/include -I/Users/Harry/.pyenv/versions/3.9.23/include/python3.9 -c extensions/_pynini.cpp -o build/temp.macosx-15.7-arm64-cpython-39/extensions/_pynini.o -std=c++17 -Wno-register -Wno-deprecated-declarations -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char -stdlib=libc++ -mmacosx-version-min=10.12
In file included from extensions/_pynini.cpp:1252:
In file included from extensions/stringmapscript.h:31:
extensions/stringmap.h:185:26: error: no member named 'StringJoin' in namespace 'fst'; did you mean 'StrJoin'?
185 | << fst::StringJoin(line, "\t") << "`";
| ~~~~~^~~~~~~~~~
| StrJoin
/opt/homebrew/opt/openfst/include/fst/compat.h:197:13: note: 'StrJoin' declared here
197 | std::string StrJoin(const std::vector<S> &elements, std::string_view delim) {
| ^
In file included from extensions/_pynini.cpp:1252:
In file included from extensions/stringmapscript.h:31:
extensions/stringmap.h:230:26: error: no member named 'StringJoin' in namespace 'fst'; did you mean 'StrJoin'?
230 | << fst::StringJoin(line, "\t") << "`";
| ~~~~~^~~~~~~~~~
| StrJoin
/opt/homebrew/opt/openfst/include/fst/compat.h:197:13: note: 'StrJoin' declared here
197 | std::string StrJoin(const std::vector<S> &elements, std::string_view delim) {
| ^
2 errors generated.
error: command '/usr/bin/clang++' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pynini
Building wheel for distance (pyproject.toml) ... done
Created wheel for distance: filename=distance-0.1.3-py3-none-any.whl size=16321 sha256=5b3994bb5686de9bbc7f78ce4f2e702b482f572e8620f0e8216d827950dabdae
Stored in directory: /Users/Harry/Library/Caches/pip/wheels/fb/b3/aa/04241cced6d1722b132273b1d6aafba317887ec004f48b853a
Building wheel for argbind (pyproject.toml) ... done
Created wheel for argbind: filename=argbind-0.3.9-py2.py3-none-any.whl size=11816 sha256=d19bcc9a580acc68e2b638466adefc2ef26058064a3e8cab62acda610b7a539d
Stored in directory: /Users/Harry/Library/Caches/pip/wheels/3e/9e/9c/0ab23ae19848099a030aaa8f68a3875e629671d859a40a7fc9
Building wheel for julius (pyproject.toml) ... done
Created wheel for julius: filename=julius-0.2.7-py3-none-any.whl size=21966 sha256=fb8693a91aed2dd06389c064ff742fdb7f6d3d722ef79f5ad2f4707e10d178ab
Stored in directory: /Users/Harry/Library/Caches/pip/wheels/53/0a/a7/fc08f97438f4969d86afa7904336c2eb7eb422101359f3ad11
Building wheel for randomname (pyproject.toml) ... done
Created wheel for randomname: filename=randomname-0.2.1-py3-none-any.whl size=89288 sha256=472a9c1becbf4931603c4affad5c5748b46dca930643b795721cc0ecf8265bae
Stored in directory: /Users/Harry/Library/Caches/pip/wheels/32/f7/3b/a46d1bb5fd36d11db2d6c267513fb405f2457cbcd6b50c760f
Successfully built jieba deepspeed antlr4-python3-runtime distance argbind julius randomname
Failed to build pynini
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> pynini

openfst 和 pynini 都安装成功了,安装requirements.txt报错关键信息如下:

1
2
3
4
5
In file included from extensions/stringmapscript.h:31:
extensions/stringmap.h:185:26: error: no member named 'StringJoin' in namespace 'fst'; did you mean 'StrJoin'?
185 | << fst::StringJoin(line, "\t") << "`";
| ~~~~~^~~~~~~~~~
| StrJoin

看信息是安装 pynini extion 时失败了,从error: no member named 'StringJoin' in namespace 'fst'; did you mean 'StrJoin'?看还是 pynini 和 openfst 不兼容的问题。在 OpenFST 1.8.0 中,StringJoin函数被重命名为StrJoin了,所以我们得修改 pynini 源码来安装了:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# 卸载当前 pynini
pip uninstall pynini -y

# 下载 pynini 源码
wget https://github.com/yanzhenjie/pynini/archive/refs/tags/v2.1.7.tar.gz
tar -xzf v2.1.7.tar.gz
cd pynini-2.1.7

# 应用修复补丁 (将 StringJoin 替换为 StrJoin)
sed -i '' 's/fst::StringJoin/fst::StrJoin/g' extensions/stringmap.h

# 安装pynini的依赖
pip install -r requirements.txt

# 重新安装修复后的 pynini
pip install .

特别注意:这里需要手动把本地 pynini 加入到 requirements.txt 中,否则还是会从远端安装。

1
2
./pynini-2.1.7
accelerate==1.8.1

从源码安装 pynini 又成功了,紧接着我又开始继续安装项目依赖pip install -r requirements.txt

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
...
Using cached wcwidth-0.2.14-py2.py3-none-any.whl (37 kB)
Building wheels for collected packages: pynini
Building wheel for pynini (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for pynini (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [68 lines of output]
/private/var/folders/wh/33kqcmn975z7bsfmvmlrsy880000gp/T/pip-build-env-o0w8wif1/overlay/lib/python3.10/site-packages/setuptools/config/_apply_pyprojecttoml.py:82: SetuptoolsDeprecationWarning: `project.license` as a TOML table is deprecated
!!

********************************************************************************
Please use a simple string containing a SPDX expression for `project.license`. You can also use `project.license-files`. (Both options available on setuptools>=77.0.0).

By 2026-Feb-18, you need to update your project and remove deprecated calls
or your builds will no longer be supported.

See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
********************************************************************************

!!
corresp(dist, value, root_dir)
/private/var/folders/wh/33kqcmn975z7bsfmvmlrsy880000gp/T/pip-build-env-o0w8wif1/overlay/lib/python3.10/site-packages/setuptools/config/_apply_pyprojecttoml.py:61: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

********************************************************************************
Please consider removing the following classifiers in favor of a SPDX license expression:

License :: OSI Approved :: Apache Software License

See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
********************************************************************************

!!
dist._finalize_license_expression()
/private/var/folders/wh/33kqcmn975z7bsfmvmlrsy880000gp/T/pip-build-env-o0w8wif1/overlay/lib/python3.10/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

********************************************************************************
Please consider removing the following classifiers in favor of a SPDX license expression:

License :: OSI Approved :: Apache Software License

See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
********************************************************************************

!!
self._finalize_license_expression()
running bdist_wheel
running build
running build_py
running egg_info
writing pynini.egg-info/PKG-INFO
writing dependency_links to pynini.egg-info/dependency_links.txt
writing top-level names to pynini.egg-info/top_level.txt
reading manifest file 'pynini.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
adding license file 'LICENSE'
adding license file 'AUTHORS'
writing manifest file 'pynini.egg-info/SOURCES.txt'
running build_ext
building '_pywrapfst' extension
clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/opt/homebrew/opt/openfst/include -I/Users/Harry/Workspace/ModelScope/IndexTTS-2-Demo/.venv/include -I/Users/Harry/.pyenv/versions/3.10.12/include/python3.10 -c extensions/_pywrapfst.cpp -o build/temp.macosx-15.7-arm64-cpython-310/extensions/_pywrapfst.o -std=c++17 -Wno-register -Wno-deprecated-declarations -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char -stdlib=libc++ -mmacosx-version-min=10.12
In file included from extensions/_pywrapfst.cpp:1193:
In file included from /opt/homebrew/opt/openfst/include/fst/fstlib.h:57:
/opt/homebrew/opt/openfst/include/fst/const-fst.h:403:25: warning: comparison of integers of different signs: 'int64_t' (aka 'long long') and 'size_t' (aka 'unsigned long') [-Wsign-compare]
403 | if (hdr.NumStates() != num_states) {
| ~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~
/opt/homebrew/opt/openfst/include/fst/const-fst.h:407:23: warning: comparison of integers of different signs: 'int64_t' (aka 'long long') and 'size_t' (aka 'unsigned long') [-Wsign-compare]
407 | if (hdr.NumArcs() != num_arcs) {
| ~~~~~~~~~~~~~ ^ ~~~~~~~~
2 warnings generated.
clang++ -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/opt/homebrew/opt/openfst/include -bundle -undefined dynamic_lookup -I/opt/homebrew/opt/openfst/include build/temp.macosx-15.7-arm64-cpython-310/extensions/_pywrapfst.o -L/Users/Harry/.pyenv/versions/3.10.12/lib -lfstfarscript -lfstfar -lfstscript -lfst -lm -ldl -o build/lib.macosx-15.7-arm64-cpython-310/_pywrapfst.cpython-310-darwin.so
ld: library 'fstfarscript' not found
clang++: error: linker command failed with exit code 1 (use -v to see invocation)
error: command '/usr/bin/clang++' failed with exit code 1
[end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pynini
Failed to build pynini
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> pynini

好家伙,还是链接错误ld: library 'fstfarscript' not found,这是由于 OpenFST 的库命名在不同 Python 版本下行为不一致导致的,查了一下可能的原因:

OpenFST 库命名变化:

  • OpenFST 1.8.0+ 版本不再提供 libfstfarscript 库
  • 它被拆分为 libfstfar 和 libfstscript 两个独立库
  • 但在 pynini 的构建配置中,仍然尝试链接 fstfarscript

所以接下来我们让-lfstfar -lfstscript替代-lfstfarscript

1
2
3
4
# 确保设置正确的 LDFLAGS
export CFLAGS="-I$(brew --prefix openfst)/include $CFLAGS"
export CPPFLAGS="-I$(brew --prefix openfst)/include $CPPFLAGS"
export LDFLAGS="-L$(brew --prefix openfst)/lib -lfstfar -lfstscript $LDFLAGS"

重新安装 pynini:

1
2
3
4
cd pynini-2.1.7
pip uninstall pynini -y
pip install -r requirements.txt
pip install .

接下来到项目根目录安装依赖pip install -r requirements.txt

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/, https://pypi.tuna.tsinghua.edu.cn/simple/, https://mirrors.ustc.edu.cn/pypi/simple/, https://pypi.org/simple/
Processing ./pynini-2.1.7
Installing build dependencies ... done
...
Using cached wcwidth-0.2.14-py2.py3-none-any.whl (37 kB)
Building wheels for collected packages: pynini
Building wheel for pynini (pyproject.toml) ... done
Created wheel for pynini: filename=pynini-2.1.7-cp310-cp310-macosx_15_0_arm64.whl size=2023566 sha256=97cba0df2440c1d0bf6c5ec910a40bc614d8e7ae41b815921dc3f0a086a68523
Stored in directory: /Users/Harry/Library/Caches/pip/wheels/81/07/cb/9d3f76d8da6b8b6791674ef8dc617e86bba48058c1165d11bb
Successfully built pynini
Installing ... spaces
Attempting uninstall: pynini
Found existing installation: pynini 2.1.7
Uninstalling pynini-2.1.7:
Successfully uninstalled pynini-2.1.7
Attempting uninstall: Cython
Found existing installation: Cython 3.1.4
Uninstalling Cython-3.1.4:
Successfully uninstalled Cython-3.1.4
Successfully installed Cython-3.0.7 ... wetext-0.1.0

成功了,✌🏻✌🏻✌🏻!!!

2.1、问题修复汇总

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# 1. 安装 OpenFST
brew install openfst

# 2. 获取 pynini 源码
wget https://github.com/yanzhenjie/pynini/archive/refs/tags/v2.1.7.tar.gz
tar -xzf v2.1.7.tar.gz
cd pynini-2.1.7

# 3. 修复 stringmap.h
sed -i '' 's/fst::StringJoin/fst::StrJoin/g' extensions/stringmap.h

# 4. 修复链接问题
export CFLAGS="-I$(brew --prefix openfst)/include $CFLAGS"
export CPPFLAGS="-I$(brew --prefix openfst)/include $CPPFLAGS"
export LDFLAGS="-L$(brew --prefix openfst)/lib -lfstfar -lfstscript $LDFLAGS"

# 5. 安装 pynini
pip uninstall pynini -y
pip install -r requirements.txt .
pip install .

# 6. 修改 requirements.txt,添加如下内容
# ./pynini-2.1.7

# 7. 在项目中安装其他依赖
cd ..
pip install -r requirements.txt

3、本地预览

在项目根目录下运行python app.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
Looking in indexes: https://mirrors.aliyun.com/pypi/simple/, https://pypi.tuna.tsinghua.edu.cn/simple/, https://mirrors.ustc.edu.cn/pypi/simple/, https://pypi.org/simple/
Collecting protobuf==3.20.3
...
modelscope - INFO - Got n files, start to download ...
...
modelscope - INFO - Download model 'IndexTeam/IndexTTS-2' successfully.
---model_id--- nvidia/bigvgan_v2_22khz_80band_256x
Loading weights from nv-community/bigvgan_v2_22khz_80band_256x
Removing weight norm...
>> bigvgan weights restored from: nvidia/bigvgan_v2_22khz_80band_256x
>> TextNormalizer loaded
>> bpe model loaded from: ./checkpoints/bpe.model
* Running on local URL: http://192.168.1.3:7860
* To create a public link, set `share=True` in `launch()`.

这里根据网络情况的不同需要几十分钟,因为要下载很多模型文件,然后看到如上提示就可以本地访问了。

 请作者来杯CAFFEE
 评论
评论插件加载失败
正在加载评论插件