Linux相机的驱动安装

Orbbec astra pro

安装 Orbbec OpenNi SDK

下载：https://orbbec3d.com/develop/ 最下面选择 "Download Orbbec OpenNi SDK"
安装，根据自己的平台选择，我这里是安装到ubuntu 18.04 64位平台，选择OpenNI-Linux-x64-2.3.0.66

cd OpenNI-Linux-x64-2.3.0.66
sudo chmod 777 install.sh
sudo ./install.sh
source OpenNIDevEnvironment

声明环境，在.bashrc文件的末尾加入下面这段话，再次source ~/.bashrc即可

#具体路径根据自己情况修改
source ~/Documents/OpenNI-Linux-x64-2.3.0.66/OpenNIDevEnvironment

安装 OpenNi Python

pip安装，我使用pip3，安装到python3

pip3 install openni

测试

from openni import openni2
import numpy as np
import cv2

def mousecallback(event,x,y,flags,param):
     if event==cv2.EVENT_LBUTTONDBLCLK:
         print(y, x, dpt[y,x])
        
if __name__ == "__main__": 
    openni2.initialize()
    dev = openni2.Device.open_any()
    print(dev.get_device_info())

    depth_stream = dev.create_depth_stream()
    depth_stream.start()

    cap = cv2.VideoCapture(0)
    cv2.namedWindow('depth')
    cv2.setMouseCallback('depth',mousecallback)

    while True:
        frame = depth_stream.read_frame()
        dframe_data = np.array(frame.get_buffer_as_triplet()).reshape([480, 640, 2])
        dpt1 = np.asarray(dframe_data[:, :, 0], dtype='float32')
        dpt2 = np.asarray(dframe_data[:, :, 1], dtype='float32')
        
        dpt2 *= 255
        dpt = dpt1 + dpt2
        
        cv2.imshow('depth', dpt)
        ret,frame = cap.read()
        cv2.imshow('color', frame)
        key = cv2.waitKey(1)
        if int(key) == ord('q'):
            break

    depth_stream.stop()
    dev.close()

注：cv2可以使用 pip3 install opencv-python安装

注：找不到设备可能是没有 source OpenNIDevEnvironment

Microsoft kinect

Linux安装kinect驱动 libfreenect2 + pylibfreenect2

原链接：fantasylsc/pylibfreenect2

Install libfreenect2 following: https://github.com/OpenKinect/libfreenect2/blob/master/README.md#linux
Set environment vatiables
add these lines to the end of "~/.bashrc":
"export LIBFREENECT2_INSTALL_PREFIX=$HOME/freenect2"
"export LD_LIBRARY_PATH= $LD_LIBRARY_PATH:$ LIBFREENECT2_INSTALL_PREFIX/lib"
Install: "pip install pylibfreenect2"

安装 libfreenect2

下载 libfreenect2 源码

git clone https://github.com/OpenKinect/libfreenect2.git
cd libfreenect2

安装依赖

注：依赖这部分有很多可选，以下是最精简的，其余选项看原链接

sudo apt-get install build-essential cmake pkg-config
sudo apt-get install libusb-1.0-0-dev
sudo apt-get install libturbojpeg0-dev
sudo apt-get install libglfw3-dev

编译

mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/freenect2  
(or cmake .. -DCMAKE_INSTALL_PREFIX=/usr/local/)
make
make install

复制规则

sudo cp ../platform/linux/udev/90-kinect2.rules /etc/udev/rules.d/

配置系统环境变量

将以下内容放在"~/.bashrc"的尾部

export LIBFREENECT2_INSTALL_PREFIX=$HOME/freenect2
(or export LIBFREENECT2_INSTALL_PREFIX=/usr/local/)
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$LIBFREENECT2_INSTALL_PREFIX/lib

安装 pylibfreenect2

注：以下使用python3 及pip3安装

pip安装

pip3 install pylibfreenect2

编译安装

pip3 install numpy
pip3 install cython
pip3 install opencv-python
直接安装最新编译版本：如下
pip3 install git+https://github.com/r9y9/pylibfreenect2
或者下载源码，再手动安装：如下
https://github.com/r9y9/pylibfreenect2.git
python3 setup.py install

注：如果出现opencv安装不动的情况，可能是pip版本过旧升级pip：pip3 install --upgrade pip

测试安装

pylibfreenect2/multiframe_listener.py at master · r9y9/pylibfreenect2

使用python3运行pylibfreenect2/examples中的multiframe_listener.py

如以上步骤均正确，则能正常看到图像

注：虚拟机不支持，运行会提示找不到设备，使用lsusb查看也没有设备接口

测试代码如下(精简版，只保留RGB与深度)：

# coding: utf-8
import numpy as np
import cv2
import sys
from pylibfreenect2 import Freenect2, SyncMultiFrameListener
from pylibfreenect2 import FrameType, Registration, Frame
from pylibfreenect2 import createConsoleLogger, setGlobalLogger
from pylibfreenect2 import LoggerLevel

try:
    from pylibfreenect2 import OpenGLPacketPipeline
    pipeline = OpenGLPacketPipeline()
except:
    try:
        from pylibfreenect2 import OpenCLPacketPipeline
        pipeline = OpenCLPacketPipeline()
    except:
        from pylibfreenect2 import CpuPacketPipeline
        pipeline = CpuPacketPipeline()
print("Packet pipeline:", type(pipeline).__name__)

fn = Freenect2()
num_devices = fn.enumerateDevices()
if num_devices == 0:
    print("No device connected!")
    sys.exit(1)

serial = fn.getDeviceSerialNumber(0)
device = fn.openDevice(serial, pipeline=pipeline)

listener = SyncMultiFrameListener(
    FrameType.Color | FrameType.Ir | FrameType.Depth)

# Register listeners
device.setColorFrameListener(listener)
device.setIrAndDepthFrameListener(listener)

device.start()
# NOTE: must be called after device.start()
registration = Registration(device.getIrCameraParams(),
                            device.getColorCameraParams())

undistorted = Frame(512, 424, 4)
registered = Frame(512, 424, 4)

while True:
    frames = listener.waitForNewFrame()

    color = frames["color"]
    depth = frames["depth"]

    registration.apply(color, depth, undistorted, registered,
                       bigdepth=bigdepth,
                       color_depth_map=color_depth_map)

    cv2.imshow("depth", depth.asarray() / 4500.)
    cv2.imshow("color", color.asarray())

    listener.release(frames)

    key = cv2.waitKey(delay=1)
    if key == ord('q'):
        break

device.stop()
device.close()

sys.exit(0)