WebRTC: Difference between revisions
Created page with "WebRTC (Web Real-Time Communication) is a standard for peer-to-peer real-time communication. It supports audio streams, video streams, and data streams but has a complicated..." |
|||
(9 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
WebRTC (Web Real-Time Communication) is a standard for peer-to-peer real-time communication. | [https://webrtc.org/ WebRTC (Web Real-Time Communication)] is a standard for peer-to-peer real-time communication. | ||
It supports audio streams, video streams, and data streams but has a complicated API and handshake which needs to be performed over an existing connection such as WebSockets to a server accessible from both peers. | It supports audio streams, video streams, and data streams but has a complicated API and handshake which needs to be performed over an existing connection such as WebSockets to a server accessible from both peers. | ||
==Background (JavaScript)== | |||
To get started, first read this background section, then look into WebRTC examples in JavaScript. | |||
WebRTC is a real-time P2P communications protocol which supports audio, video, and data. | |||
For P2P applications to work, they need to first negotiation a connection using an intermediary server. | |||
This signalling is typically done through a WebSocket but can also be done using AJAX via Socket.IO. | |||
==Native APIs (C++)=== | ===MediaStream=== | ||
The purpose of MediaStream to to access the user's camera, microphone, or screen. | |||
This is done using [https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia MediaDevices.getUserMedia] | |||
===RTCPeerConnection=== | |||
The purpose of RTCPeerConnection is to negotiate a connection using the signalling channel (the Websocket forwarded through your nat). | |||
When you make an RTCPeerConnection, you should pass in a list of iceServers: | |||
<syntaxhighlight lang="javascript"> | |||
const configuration = {iceServers: [{urls: 'stun.l.google.com:19302'}]}; | |||
const pc = new RTCPeerConnection(configuration); | |||
</syntaxhighlight> | |||
There are two types of iceServers: | |||
* STUN servers are used to find the public IP and port of the client. | |||
** You can find a list of STUN servers [https://gist.github.com/mondain/b0ec1cf5f60ae726202e here]. | |||
* TURN servers are a fallback used to proxy data through the NAT. | |||
For each STUN server, you need to send it to the other peer via your signalling channel: | |||
<syntaxhighlight lang="javascript"> | |||
const configuration = {iceServers: [{urls: 'stuns:stun.example.org'}]}; | |||
</syntaxhighlight> | |||
The other client will add these as follows: | |||
<syntaxhighlight lang="javascript"> | |||
remoteConnection.addIceCandidate(e.candidate) | |||
</syntaxhighlight> | |||
Next the local and remote clients exchange descriptions: | |||
<syntaxhighlight lang="javascript"> | |||
const localOffer = await localConnection.createOffer(); | |||
localConnection.setLocalDescription(localOffer); | |||
signallingChannel.send(localConnection.localDescription) | |||
# On the remote | |||
const localOffer = signalingChannel.receiveAnswer(); // Actually a callback irl | |||
remoteConnection.setRemoteDescription(localOffer); | |||
const answer = await remoteConnection.createAnswer(); | |||
remoteConnection.setLocalDescription(answer); | |||
signallingChannel.send(remoteConnection.localDescription) | |||
# On the local | |||
const remoteDescription = signalingChannel.receiveAnswer(); | |||
localConnection.setRemoteDescription(remoteDescription); | |||
</syntaxhighlight> | |||
* Note that both clients can continue to update descriptions as the network updates. | |||
*: As long as the signalling is setup correctly, you don't need to worry about what RTCPeerConnection is doing under the hood. | |||
{{ hidden | RTCPeerConnection Example | | |||
Below is from [https://www.html5rocks.com/en/tutorials/webrtc/basics/ HTML5Rocks]. | |||
<syntaxhighlight lang="javascript"> | |||
// handles JSON.stringify/parse | |||
const signaling = new SignalingChannel(); | |||
const constraints = {audio: true, video: true}; | |||
const configuration = {iceServers: [{urls: 'stuns:stun.example.org'}]}; | |||
const pc = new RTCPeerConnection(configuration); | |||
// send any ice candidates to the other peer | |||
pc.onicecandidate = ({candidate}) => signaling.send({candidate}); | |||
// let the "negotiationneeded" event trigger offer generation | |||
pc.onnegotiationneeded = async () => { | |||
try { | |||
await pc.setLocalDescription(await pc.createOffer()); | |||
// send the offer to the other peer | |||
signaling.send({desc: pc.localDescription}); | |||
} catch (err) { | |||
console.error(err); | |||
} | |||
}; | |||
// once remote track media arrives, show it in remote video element | |||
pc.ontrack = (event) => { | |||
// don't set srcObject again if it is already set. | |||
if (remoteView.srcObject) return; | |||
remoteView.srcObject = event.streams[0]; | |||
}; | |||
// call start() to initiate | |||
async function start() { | |||
try { | |||
// get local stream, show it in self-view and add it to be sent | |||
const stream = | |||
await navigator.mediaDevices.getUserMedia(constraints); | |||
stream.getTracks().forEach((track) => | |||
pc.addTrack(track, stream)); | |||
selfView.srcObject = stream; | |||
} catch (err) { | |||
console.error(err); | |||
} | |||
} | |||
signaling.onmessage = async ({desc, candidate}) => { | |||
try { | |||
if (desc) { | |||
// if we get an offer, we need to reply with an answer | |||
if (desc.type === 'offer') { | |||
await pc.setRemoteDescription(desc); | |||
const stream = | |||
await navigator.mediaDevices.getUserMedia(constraints); | |||
stream.getTracks().forEach((track) => | |||
pc.addTrack(track, stream)); | |||
await pc.setLocalDescription(await pc.createAnswer()); | |||
signaling.send({desc: pc.localDescription}); | |||
} else if (desc.type === 'answer') { | |||
await pc.setRemoteDescription(desc); | |||
} else { | |||
console.log('Unsupported SDP type.'); | |||
} | |||
} else if (candidate) { | |||
await pc.addIceCandidate(candidate); | |||
} | |||
} catch (err) { | |||
console.error(err); | |||
} | |||
}; | |||
</syntaxhighlight> | |||
}} | |||
===RTCDataChannel=== | |||
<syntaxhighlight lang="javascript"> | |||
const localConnection = new RTCPeerConnection(servers); | |||
const remoteConnection = new RTCPeerConnection(servers); | |||
const sendChannel = | |||
localConnection.createDataChannel('sendDataChannel'); | |||
// ... | |||
remoteConnection.ondatachannel = (event) => { | |||
receiveChannel = event.channel; | |||
receiveChannel.onmessage = onReceiveMessage; | |||
receiveChannel.onopen = onReceiveChannelStateChange; | |||
receiveChannel.onclose = onReceiveChannelStateChange; | |||
}; | |||
function onReceiveMessage(event) { | |||
document.querySelector("textarea#send").value = event.data; | |||
} | |||
document.querySelector("button#send").onclick = () => { | |||
var data = document.querySelector("textarea#send").value; | |||
sendChannel.send(data); | |||
}; | |||
</syntaxhighlight> | |||
==JavaScript API== | |||
===Multiple Clients=== | |||
To handle multiple clients, you simply have one RTCPeerConnection per client. | |||
==Native APIs (C++)== | |||
==Resources== | |||
* [https://www.html5rocks.com/en/tutorials/webrtc/basics/ HTML5Rocks: Getting Started with WebRTC (JS)] | |||
* [https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Simple_RTCDataChannel_sample MDN Simple RTCDataChannel sample] | |||
* [https://news.ycombinator.com/item?id=25933016 HackerNews: WebRTC is now a W3C and IETF standard] | |||
==References== |
Latest revision as of 14:12, 28 January 2021
WebRTC (Web Real-Time Communication) is a standard for peer-to-peer real-time communication.
It supports audio streams, video streams, and data streams but has a complicated API and handshake which needs to be performed over an existing connection such as WebSockets to a server accessible from both peers.
Background (JavaScript)
To get started, first read this background section, then look into WebRTC examples in JavaScript.
WebRTC is a real-time P2P communications protocol which supports audio, video, and data.
For P2P applications to work, they need to first negotiation a connection using an intermediary server.
This signalling is typically done through a WebSocket but can also be done using AJAX via Socket.IO.
MediaStream
The purpose of MediaStream to to access the user's camera, microphone, or screen. This is done using MediaDevices.getUserMedia
RTCPeerConnection
The purpose of RTCPeerConnection is to negotiate a connection using the signalling channel (the Websocket forwarded through your nat).
When you make an RTCPeerConnection, you should pass in a list of iceServers:
const configuration = {iceServers: [{urls: 'stun.l.google.com:19302'}]};
const pc = new RTCPeerConnection(configuration);
There are two types of iceServers:
- STUN servers are used to find the public IP and port of the client.
- You can find a list of STUN servers here.
- TURN servers are a fallback used to proxy data through the NAT.
For each STUN server, you need to send it to the other peer via your signalling channel:
const configuration = {iceServers: [{urls: 'stuns:stun.example.org'}]};
The other client will add these as follows:
remoteConnection.addIceCandidate(e.candidate)
Next the local and remote clients exchange descriptions:
const localOffer = await localConnection.createOffer();
localConnection.setLocalDescription(localOffer);
signallingChannel.send(localConnection.localDescription)
# On the remote
const localOffer = signalingChannel.receiveAnswer(); // Actually a callback irl
remoteConnection.setRemoteDescription(localOffer);
const answer = await remoteConnection.createAnswer();
remoteConnection.setLocalDescription(answer);
signallingChannel.send(remoteConnection.localDescription)
# On the local
const remoteDescription = signalingChannel.receiveAnswer();
localConnection.setRemoteDescription(remoteDescription);
- Note that both clients can continue to update descriptions as the network updates.
- As long as the signalling is setup correctly, you don't need to worry about what RTCPeerConnection is doing under the hood.
Below is from HTML5Rocks.
// handles JSON.stringify/parse
const signaling = new SignalingChannel();
const constraints = {audio: true, video: true};
const configuration = {iceServers: [{urls: 'stuns:stun.example.org'}]};
const pc = new RTCPeerConnection(configuration);
// send any ice candidates to the other peer
pc.onicecandidate = ({candidate}) => signaling.send({candidate});
// let the "negotiationneeded" event trigger offer generation
pc.onnegotiationneeded = async () => {
try {
await pc.setLocalDescription(await pc.createOffer());
// send the offer to the other peer
signaling.send({desc: pc.localDescription});
} catch (err) {
console.error(err);
}
};
// once remote track media arrives, show it in remote video element
pc.ontrack = (event) => {
// don't set srcObject again if it is already set.
if (remoteView.srcObject) return;
remoteView.srcObject = event.streams[0];
};
// call start() to initiate
async function start() {
try {
// get local stream, show it in self-view and add it to be sent
const stream =
await navigator.mediaDevices.getUserMedia(constraints);
stream.getTracks().forEach((track) =>
pc.addTrack(track, stream));
selfView.srcObject = stream;
} catch (err) {
console.error(err);
}
}
signaling.onmessage = async ({desc, candidate}) => {
try {
if (desc) {
// if we get an offer, we need to reply with an answer
if (desc.type === 'offer') {
await pc.setRemoteDescription(desc);
const stream =
await navigator.mediaDevices.getUserMedia(constraints);
stream.getTracks().forEach((track) =>
pc.addTrack(track, stream));
await pc.setLocalDescription(await pc.createAnswer());
signaling.send({desc: pc.localDescription});
} else if (desc.type === 'answer') {
await pc.setRemoteDescription(desc);
} else {
console.log('Unsupported SDP type.');
}
} else if (candidate) {
await pc.addIceCandidate(candidate);
}
} catch (err) {
console.error(err);
}
};
RTCDataChannel
const localConnection = new RTCPeerConnection(servers);
const remoteConnection = new RTCPeerConnection(servers);
const sendChannel =
localConnection.createDataChannel('sendDataChannel');
// ...
remoteConnection.ondatachannel = (event) => {
receiveChannel = event.channel;
receiveChannel.onmessage = onReceiveMessage;
receiveChannel.onopen = onReceiveChannelStateChange;
receiveChannel.onclose = onReceiveChannelStateChange;
};
function onReceiveMessage(event) {
document.querySelector("textarea#send").value = event.data;
}
document.querySelector("button#send").onclick = () => {
var data = document.querySelector("textarea#send").value;
sendChannel.send(data);
};
JavaScript API
Multiple Clients
To handle multiple clients, you simply have one RTCPeerConnection per client.
Native APIs (C++)
Resources
- HTML5Rocks: Getting Started with WebRTC (JS)
- MDN Simple RTCDataChannel sample
- HackerNews: WebRTC is now a W3C and IETF standard