JavaCV 采集摄像头和麦克风数据推送到流媒体服务器

越来越觉得放弃JavaCV FFmpeg native API，直接使用JavaCV二次封装的API开发是很明智的选择，使用JavaCV二次封装的API开发避免了各种内存操作不当引起的crash。

上一次介绍了 JavaCV 采集摄像头及桌面视频数据，这次介绍一下如何采集摄像头和麦克风数据推送到流媒体服务器。

引入依赖

跟上一次一样，这里使用的还是最新的JavaCV库（1.5.5）.

<properties>
  <javacpp.version>1.5.5</javacpp.version>
</properties>
<dependencies>
  <dependency>
    <groupId>org.bytedeco</groupId>
    <artifactId>javacv</artifactId>
    <version>${javacpp.version}</version>
  </dependency>
  <dependency>
    <groupId>org.bytedeco</groupId>
    <artifactId>javacv-platform</artifactId>
    <version>${javacpp.version}</version>
  </dependency>
</dependencies>

视频采集录制

在 JavaCV 采集摄像头及桌面视频数据 介绍了两种采集摄像头数据的方法，这里采用第一种，即使用 OpencvFrameGrabber 采集摄像头数据。

public class VideoRecorder implements Runnable {

  private static final int VIDEO_DEVICE_INDEX = 0;
  private FFmpegFrameRecorder recorder;
  private int width, height;
  public VideoRecorder(FFmpegFrameRecorder recorder, int width, int height) {
    this.recorder = recorder;
    this.width = width;
    this.height = height;
  }

  @Override
  public void run() {
    try {
      OpenCVFrameGrabber grabber = new OpenCVFrameGrabber(VIDEO_DEVICE_INDEX);
      grabber.setImageWidth(width);
      grabber.setImageHeight(height);
      grabber.start();

      long startTS = 0, videoTS = 0;
      Frame frame = null;
      while (!Thread.interrupted() && (frame = grabber.grab()) != null) {
        if (startTS == 0) {
          startTS = System.currentTimeMillis();
        }
        videoTS = 1000 * (System.currentTimeMillis() - startTS);
        if (videoTS > recorder.getTimestamp()) {
          recorder.setTimestamp(videoTS);
        }
        recorder.record(frame);
      }

      grabber.close();
    } catch (Exception e) {
      e.printStackTrace();
    }
  }
}

音频采集录制

音频采集直接使用JDK的API即可，直接读麦克风的数据塞给录制器，采样率为44100，16bit，小端模式。

public class AudioRecoder implements Runnable {

  private FFmpegFrameRecorder recorder;
  private int channels;
  private int sampleRate;
  public AudioRecoder(FFmpegFrameRecorder recorder, int sampleRate, int channels) {
    this.recorder = recorder;
    this.sampleRate = sampleRate;
    this.channels = channels;
  }

  @Override
  public void run() {
    try {
      AudioFormat format = new AudioFormat(Float.valueOf(sampleRate), 16, channels, true, false);
      TargetDataLine line = (TargetDataLine) AudioSystem.getLine(new DataLine.Info(TargetDataLine.class, format));
      line.open(format);
      line.start();

      int sampleRate = (int) format.getSampleRate();
      int numChannels = format.getChannels();
      byte[] buffer = new byte[sampleRate * numChannels];

      while (!Thread.interrupted()) {
        int nBytesRead = 0;
        while (nBytesRead == 0) {
          nBytesRead = line.read(buffer, 0, line.available());
        }
        int nSamplesRead = nBytesRead / 2;
        short[] samples = new short[nSamplesRead];
        ByteBuffer.wrap(buffer).order(ByteOrder.LITTLE_ENDIAN).asShortBuffer().get(samples);
        ShortBuffer sBuff = ShortBuffer.wrap(samples, 0, nSamplesRead);
        recorder.recordSamples(sampleRate, numChannels, sBuff);
      }
    } catch (Exception e) {
      e.printStackTrace();
    }
  }

}

录制推流

这里的录制推流涉及音频和视频，音频和视频分开线程采集并录制，这样可以降低延迟，在录制推流方面还可以通过设置录制参数降低延迟：

recorder.setVideoOption("tune", "zerolatency");
recorder.setVideoOption("preset", "ultrafast");

启动录制推流器，视频采集录制线程和音频采集录制线程，开始推流：

recorder.start();
Thread vt = new Thread(new VideoRecorder(recorder, width, height));
Thread at = new Thread(new AudioRecoder(recorder, sampleRate, channels));

vt.start();
at.start();

vt.join();
at.join();

效果展示

可以使用VLC拉取程序的推流查看效果：

拉流的音视频流信息：

录制推流完整源码可以在公众号上获取

=========================================================

关注公众号 “HiIT青年” 发送 “javacv-recoder” 获取。（如果没有收到回复，可能是你之前取消过关注。）

关注公众号，阅读更多文章。