Skip to content

About

一个跨平台 .NET 全功能音频处理库,面向 .NET 开发者提供编解码、设备 I/O、DSP 效果、语音处理、容器格式读写、流媒体传输等一站式音频能力。

Resources

Stars

5 stars

Watchers

10 watching

Forks

Latest commit

 

History

309 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

NewLife.Audio - Audio Codec Library

GitHub top language

GitHub License NuGet Downloads NuGet NuGet prerelease test NativeAOT

AOT smoke TFM locales

platform arch contributors

🌐 简体中文 | 繁體中文 | Español | Deutsch | 日本語 | Français | Português | Русский | 한국어 | العربية

简体中文 | English

A cross-platform, pure-managed .NET audio processing library supporting codecs, device I/O, DSP effects, speech processing, text-to-speech, speech recognition, audio file read/edit/write, container format I/O, streaming, and MIDI sequencing/rendering — all in one NuGet package.

🎧 Bundled AudioStudio cross-platform desktop demo client (Avalonia) with eight tool modules: playback, MIDI sequencing, conversion, devices, recording, DSP, diagnostics, and speech synthesis.

Source: https://github.com/NewLifeX/NewLife.Audio
NuGet: NewLife.Audio

📖 Detailed documentation (Chinese): Requirements | Feature module list | Architecture | Client requirements | Client feature list | Competitive analysis | AOT compatibility | Localization | Module architecture designs

Features

🎵 Codec Engine (CODEC)

Bidirectional PCM conversion for 10+ audio codecs, all in pure managed C#:

  • G.711 μ-law / A-law
  • G.722 / G.726 sub-band ADPCM
  • IMA ADPCM (DVI4)
  • Opus (self-hosted pure-managed implementation in progress: RFC 6716 packet parser + range coder, with ffmpeg fallback)
  • MP3 (decode: Huffman + inverse quantization + IMDCT + subband synthesis, bit reservoir, bad-frame concealment and CRC validation; encode: polyphase analysis + MDCT + quantization + Huffman, with joint stereo M/S, CBR/VBR/ABR + Xing info frame, a simplified psychoacoustic model, cross-frame bit reservoir and optional CRC; short blocks not yet implemented)
  • AAC (pure-managed AAC-LC/HE-AAC decoding; AAC-LC encoding; ADTS/raw/MP4/LOAS/LATM)
  • FLAC (Fixed LPC + Rice residual coding + MD5)
  • Ogg Vorbis (pure-managed decode: floor 0/1, residue 0/1/2, channel coupling, MDCT; baseline long-block encoder)
  • Speex (pure managed narrowband, 20 ms frames + 4-bit adaptive differential coding; private bitstream, not libspeex-compatible)
  • Pluggable factory via CodecRegistry + ICodecInfo metadata
  • IoT chip header adaptation (Hisilicon / Dahua / Uniview)

🔊 Audio Device I/O (DEVIO)

Cross-platform audio device abstraction:

  • Device enumeration & discovery (including shared-mode mix format and device interface friendly name)
  • WaveOut / WASAPI backends on Windows (ASIO planned)
  • Audio capture & loopback recording
  • Endpoint volume & mute control (WASAPI)
  • Multi-device concurrent management via AudioDeviceManager

⚠️ On Windows, both WinMM (WaveOut/WaveIn) and WASAPI (shared-mode playback/capture, loopback, event-driven low-latency streams, device-loss recovery, hot-plug watcher) are fully implemented. ASIO and the Linux/macOS/Android/iOS backends are not; their status and roadmap are documented in the architecture design.

🎛️ DSP Pipeline (DSP)

Composable audio signal processing chain:

  • Resampling (linear + windowed-sinc)
  • Channel conversion (mono ↔ stereo ↔ multichannel)
  • Bit depth / sample format conversion (8/16/24/32-bit ↔ float/double)
  • Volume control & gain (linear / logarithmic)
  • Multi-source mixing with normalization
  • FFT spectrum analysis (Cooley-Tukey, Hann/Hamming/Blackman windows)
  • Equalizer (10-band graphic + 3-band tone, BiQuad cascade)
  • BiQuad filter (RBJ standard: LP/HP/BP/Notch/Peaking/LowShelf/HighShelf)
  • Dynamic range compressor (soft-knee, RMS envelope)
  • Fade in/out & crossfade (linear/log/cosine)
  • Speed change (OLA) & pitch shift
  • Chainable IAudioProcessor + AudioPipeline

🗣️ Speech Processing (VOP)

Real-time voice communication front-end:

  • VAD — Voice Activity Detection (GMM model, 6 sub-bands + hangover)
  • AEC — Acoustic Echo Cancellation (NLMS adaptive filter + double-talk detection + comfort noise)
  • AGC — Automatic Gain Control (slow-attack fast-release, RMS envelope)
  • NS — Noise Suppression (spectral subtraction + FFT/IFFT + OLA synthesis)
  • Voice preprocessor pipeline: HPF → VAD → AGC
  • SpeechProcessingPipeline: 10 ms frame alignment, reference delay compensation, AEC → NS → AGC with ERLE/double-talk/convergence metrics

📦 Container Formats (CONT)

Audio file format read/write:

  • WAV — Full RIFF parsing & writing (EXTENSIBLE/float/24-bit/RF64/LIST-INFO)
  • Ogg — Page-level read/write (Opus/Vorbis)
  • MP3 — Frame index, ID3v2.2/v2.3/v2.4 + v1 (unsync, compressed frames, in-place tag rewrite), Xing/Info/VBRI, gapless
  • MP4/M4A — ISO BMFF: ftyp/moov/mdat box parsing & generation, iTunes tags
  • FLAC — Native fLaC metadata read/write (STREAMINFO/SEEKTABLE/VORBIS_COMMENT)
  • AIFF/CAF — AIFF read/write; CAF read
  • Probe — Magic-byte detection with container/codec/duration/bitrate/tags
  • Raw PCM — Explicit format required; no heuristic auto-detection

🗂️ Audio File Facade (FILE)

One call chain for open → decode → edit → save:

  • AudioFile.Open auto-detects container/codec and exposes format, duration, tags and seekability
  • Lossless PCM editing: trim, concat, gain, normalize, resample
  • Metadata preservation, tag-only atomic rewrite, atomic save via temp file
  • Frame-level copy/remux for MP3/AAC/WAV/Ogg Opus when parameters match; explicit re-encode fallback
  • Batch conversion with progress callbacks and cancellation
  • StreamPcm streams decoded 16-bit PCM chunks: frame-by-frame for WAV/AIFF/CAF/M4A, whole-file decode then chunking for MP3/FLAC/Ogg (memory scales with chunkBytes, not file size, for the streaming path)
  • AudioRecorder one-line recording: Capture / RecordToFile / StreamAsync, bounded queue with drop statistics
  • AudioPlayer one-line playback: file/PCM/stream, volume, backpressure, Seek and PCM observation events
  • AudioConverter one-line transcoding: container by extension, per-frame resample/channel conversion/encode, progress, cancellation and atomic replace

📡 Streaming & Transport (STRM)

Network audio streaming:

  • RTP packetization & depacketization (RFC 3550) + session management
  • RTSP client (TCP: OPTIONS→DESCRIBE→SETUP→PLAY→TEARDOWN, SDP parsing)
  • HTTP audio streaming (Icecast/SHOUTcast)
  • WebSocket bidirectional audio pipe (5-byte frame header)

🎼 MIDI Sequencing (MIDI)

Pure-managed Standard MIDI File read/write and waveform rendering:

  • Parse/write SMF 1.0 (Format 0/1/2, PPQ/SMPTE, VLQ, running status, Meta/SysEx passthrough); Format 2 sequences render independently
  • Pure-managed synthesizer: GM program groups, band-limited wavetables, ADSR, velocity/volume/expression/pan/pitch bend/sustain, polyphony limit, soft limiting
  • Optional SoundFont 2 (.sf2) and DLS Level 1/2 (.dls) sampling via MidiRenderOptions.SoundBank
  • MPE continuous pitch bend/channel pressure and MIDI 2.0 UMP packet/SysEx7 parsing
  • Windows real-time MIDI output (WinMM midiOut/midiIn, device enumeration, tempo-based MidiOutputSequencer)
  • Render to 16-bit PCM and reuse AudioFile.WritePcm for WAV/MP3/FLAC
  • AudioStudio MIDI Sequencing page: gradient min/max waveform, RMS line, beat grid, glowing playhead, read-only mini piano roll, sound-bank loading, MIDI output panel
  • Built-in public-domain demos (Ode to Joy, Greensleeves, Canon in D, Jasmine Flower) under AudioStudio/Assets/Midi
  • Console demo Samples/MidiDemo: info / render / play / gen, with optional .sf2/.dls source
  • MIDI is a score/event stream, not sampled audio; USB-MIDI/BLE-MIDI and native MIDI 2.0 transport are out of scope

🔈 Text-to-Speech (TTS)

Turn text into sound with three lines of code:

  • Speak directly — await new TtsClient().SayAsync("text") via Windows SAPI / macOS say / Linux espeak-ng (on Windows .NET Core/.NET 5+, reference the System.Speech package)
  • Save to file — WAV / raw PCM / MP3 / FLAC, chosen by file extension (compressed formats go through the resample → encode → container pipeline)
  • Stream out — PCM chunks for HTTP/WebSocket generate-as-you-send
  • Pluggable engines — system speech, HttpTtsEngine, WebSocketTtsEngine, ProcessTtsEngine, custom ITtsEngine
  • Cloud providers — AliyunTtsEngine / TencentTtsEngine / BaiduTtsEngine / AzureTtsEngine, plus key-free BaiduTranslateTtsEngine
  • Offline neural voices — optional NewLife.Audio.Sherpa package (sherpa-onnx: VITS / Matcha / Kokoro, streaming)
  • Optional advanced — voice/rate plus a voice catalog (TtsVoiceCatalog: pick voices across engines by language and scenario), disk cache, pre-generated phrases, text normalization (NewLife.Audio.Tts.Advanced)
  • LAN announcer — Samples/TtsNotify listens on a UDP port and speaks any text broadcast on the LAN (and can send too): handy for build/burn result announcements

🎙️ Speech Recognition (ASR)

Low-priority adapter layer — no bundled model or inference runtime:

  • ISpeechRecognizer streaming PCM input with partial/final results, timestamps and endpoint detection
  • Windows system recognition via System.Speech.Recognition
  • Optional NewLife.Audio.Sherpa package for offline sherpa-onnx recognition
  • Android/iOS system adapters planned with platform TFMs

Installation

dotnet add package NewLife.Audio

Quick Start

using NewLife.Audio;
using NewLife.Audio.Codecs;
using NewLife.Audio.Tts;

// Decode G.711 μ-law to PCM
var codec = new G711UCodec();
var pcm = codec.ToPcm(encodedData);

// Encode PCM to G.711 A-law
var g711a = new G711ACodec();
var encoded = g711a.FromPcm(pcmData);

// Text-to-speech: speak and save to WAV
var tts = new TtsClient();
await tts.SayAsync("Hello, world");
await tts.SaveAsync("Hello, world", "hello.wav");

Ecosystem Map

NewLife is a layered ecosystem of independently versioned open-source projects. The map below is shared verbatim by every NewLife repository; each project links to its repository and its primary NuGet package.

Project What it does Downloads
Foundation
NewLife.Core Logging, configuration, caching, networking, serialization, scheduling and APM tracing. NuGet
NewLife.XCode Data middleware: single tables at the tens-of-billions scale, automatic sharding, read-write splitting, 10+ databases. NuGet
NewLife.Net High-performance network library: 1.77M pps / 14.5 Gbps on a single machine (TCP 1KB, 2026-10), 4M long-lived TCP connections. NuGet
NewLife.Remoting Protocol communication and RPC framework for client/server applications and IoT devices. NuGet
NewLife.Agent Installs .NET applications as OS daemons: Windows services and Linux systemd. NuGet
NewLife.Templates dotnet new project templates for console, web, API, service, data and IoT projects. NuGet
Data & Messaging
NewLife.Redis Redis client with microsecond latency, millions of ops/s and built-in message queues. NuGet
NewLife.RocketMQ Pure managed RocketMQ client for Apache RocketMQ and Alibaba Cloud Message Queue. NuGet
NewLife.MQTT MQTT client and server, including Alibaba Cloud IoT connectivity. NuGet
NewLife.NovaDb Hybrid database in C#: embedded or networked, relational, time-series, message queue and KV in one. NuGet
NewLife.MySql MySql driver and helper layer built on the NewLife data stack. NuGet
Web, AI & Distributed
NewLife.Cube Rapid admin-platform framework: permissions, SSO, OAuth server and CRUD scaffolding. NuGet
Stardust Distributed service platform: node management, APM, configuration center, registry and release center. NuGet
AntJob Distributed task and real-time data scheduling platform with sharding, retries and backfill. NuGet
NewLife.AI Unified client for 46 LLM providers plus ChatAI, a ready-to-run web chat application. NuGet
Industrial & IoT
NewLife.IoT IoT standard library defining protocol contracts and device abstractions. NuGet
NewLife.Modbus Modbus TCP/RTU/ASCII implementation on top of NewLife.IoT. NuGet
NewLife.Siemens Siemens PLC protocol on top of NewLife.IoT. NuGet
NewLife.Omron Omron PLC protocol on top of NewLife.IoT. NuGet
NewLife.Melsec Mitsubishi Melsec PLC protocol on top of NewLife.IoT. NuGet
NewLife.OPC OPC client for industrial data access. —
NewLife.BACnet BACnet building-automation protocol stack. NuGet
ZeroIoT Lightweight open-source IoT platform with gateway and device management. —
Vehicle Telematics
NewLife.JT808 JT/T 808 vehicle positioning protocol: 70+ message bodies, 2011/2013/2019. NuGet
NewLife.JT809 JT/T 809 platform-to-platform protocol for transport authorities. NuGet
NewLife.JT1078 JT/T 1078 in-vehicle video: RTP parsing, FLV/fMP4 remuxing and live streaming. NuGet
Domain Components
NewLife.Office One package for Excel, Word, PPT, PDF and 15+ document formats. NuGet
NewLife.Audio Cross-platform audio: codecs, device I/O, DSP, containers and streaming. NuGet
NewLife.Video Video codecs, container formats, frame processing and streaming. NuGet
NewLife.Map Unified interface over Baidu, AMap, Tencent and Tianditu with geocoding and coordinate conversion. NuGet
NewLife.IP IPv4 geolocation with an offline memory-mapped database and an online API fallback. NuGet
Tools & Products
CrazyCoder Developer toolbox: network, serial port, crypto, regex, Modbus and MQTT. NuGet
GitCandy Self-hosted Git source code management system. —
LuckyClover Installs the matching .NET runtime automatically when launched from an older host. —

Badges show the primary package of each project; theme, extension and satellite packages are listed on their NuGet pages.The 30+ packages not listed here (FTP, LoRa, Office extensions, protocol drivers and more) are available under the NewLifeX organization and on newlifex.com.

Team

NewLife

Founded in 2002, the NewLife team has spent more than two decades on the .NET and IoT ecosystem, maintains nearly 100 open-source projects (34 of them core to this map), and delivers integrated software and hardware solutions for power, logistics, healthcare, industrial control and internet industries. We live by the motto "learning has no seniority — mastery earns respect".

Open-source components have been downloaded 6.48M times from NuGet (ecosystem-wide, as of 2026-10).

Website: https://newlifex.com · Open source organization: https://github.com/NewLifeX

WeChat Official Account: 智能大石头 (scan to follow)

智能大石头

License

MIT License © 2002-2026 NewLife Team

Third-party components (SharpJaad.AAC, EggEncoder MDCT, minimp3 tables) are listed in THIRD-PARTY-NOTICES.md.

GitHub | Website

About

一个跨平台 .NET 全功能音频处理库,面向 .NET 开发者提供编解码、设备 I/O、DSP 效果、语音处理、容器格式读写、流媒体传输等一站式音频能力。

Resources

Stars

5 stars

Watchers

10 watching

Forks

Releases

Packages

Used by

Contributors

Languages