Android calls UI

[adbenitez]

• wait for add call ringing API chatmail/core#6650 to be merged
• integrate https://github.com/deltachat/calls-webapp
• use new core calls API
• add new calls icon to title bar
• remove old videocall invitations
• wait for a new core to be tagged including the new calls API
• add ringing notification on incoming call
• make sure, multidevice is working: calls may ring on several devices, once accepted on one, the others should stop ringing
• update changelog

counterpart of deltachat/deltachat-ios#2638

[adbenitez]

the dialog that appears when inviting someone says that you need to have an app installed, we probably need a new string

[github-actions]

To test the changes in this pull request, install this apk:
📦 app-preview.apk

[r10s]

the dialog that appears when inviting someone says that you need to have an app installed, we probably need a new string

for now, we can just not use the string videochat_invite_user_hint

[r10s]

great that you figured out all that!

if needed, we can merge that roughly as is to move forward.

the only thing we should necessarily change is to not show the call option by default, maybe we need a switch for that, as we introduced a default string, but any other way that comes in mind would do as well

[github-actions]

To test the changes in this pull request, install this apk:
📦 app-preview.apk

[github-actions]

To test the changes in this pull request, install this apk:
📦 app-preview.apk

[WofWca]

Regarding the "the other participant's video freezing" issue:
I think I have found the suspect: it might be the thing where Android (or is it Chromium) pauses all media when you're on a call (or receive an "incoming call" notification?)

If you go to "saved messages" and start a call there, and then open your browser and try to play another video, you'll notice that it behaves in the same way. If you try to start playing the video, it will play for a moment, and then pause again. This is also the case for the incoming stream in our "calls" app.
Then, if you forcefully stop Delta Chat, you are able to play the media again.

However, I honestly have no idea why you can still play the video of your own camera inside the app, even when it's unmuted.

I have been debugging this with Eruda. Here is my little debug script, if you're interested:

Script

// Hide the "Ringing..." div that covers the rest of the app.
document.querySelectorAll('div')[4].style.display =''

vids = document.querySelectorAll('video')

// Having this makes the new `localVideo` play.
if (1) {
	vids[0].srcObject = null
	vids[1].srcObject = null
	vids[0].remove()
	vids[1].remove()
}

// This makes both videos play.
const enableAudio = 1
const setSrcObjectOnEventIndex = 1

let localVideo
if (1) {
	localVideo = document.createElement('video');
	localVideo.autoplay = true;
	localVideo.muted = true;
	localVideo.controls = true;
	localVideo.style.width = '320px';
	localVideo.style.marginRight = '10px';

	document.body.appendChild(localVideo);
} else {
	localVideo = vids[1]	
}

let remoteVideo
if (1) {
	remoteVideo = document.createElement('video');
	remoteVideo.autoplay = true;
	remoteVideo.controls = true;
	remoteVideo.style.width = '320px';

	document.body.appendChild(remoteVideo);
} else {
	remoteVideo = vids[0]
	remoteVideo.controls = true
}

let localStream;
const pc1 = new RTCPeerConnection();
const pc2 = new RTCPeerConnection();

// When pc2 receives a track, display it in remoteVideo
let onTrackCount = 0
pc2.ontrack = (event) => {
	onTrackCount++
  if (onTrackCount !== setSrcObjectOnEventIndex) {
	  console.log('onTrackCount !== setSrcObjectOnEventIndex')
	  return
  }
  remoteVideo.srcObject = event.streams[0];
};

// Get local media and start the connection
navigator.mediaDevices.getUserMedia({ video: true, audio: enableAudio }).then(stream => {
  localStream = stream;
  localVideo.srcObject = localStream;

	if (0) {
		return
	}

  // Add local tracks to pc1
  localStream.getTracks().forEach(track => {
    pc1.addTrack(track, localStream);
  });

  // ICE candidates
  pc1.onicecandidate = e => {
    if (e.candidate) pc2.addIceCandidate(e.candidate);
  };
  pc2.onicecandidate = e => {
    if (e.candidate) pc1.addIceCandidate(e.candidate);
  };

  // Start offer/answer exchange
  pc1.createOffer().then(offer => {
    return pc1.setLocalDescription(offer);
  }).then(() => {
    return pc2.setRemoteDescription(pc1.localDescription);
  }).then(() => {
    return pc2.createAnswer();
  }).then(answer => {
    return pc2.setLocalDescription(answer);
  }).then(() => {
    return pc1.setRemoteDescription(pc2.localDescription);
  });
});

[WofWca]

This seems to mean that you could call anyone except contacts previously.

[WofWca]

the thing where Android (or is it Chromium) pauses all media when you're on a call

Or this might be this thing called "audio focus": https://developer.android.com/media/optimize/audio-focus

[adbenitez]

in fact I tried disabling the use of the calls integration and now the video works, a bit weird because this used to work in the past, but I think for now I will not use the Android Telecom API at all, it is almost useless, only useful for calling from cars or Bluetooth devices

[link2xt]

This gets the answer from the network and injects it directly into javascript, may result in executing anything in the webview.

[link2xt]

SDP sent over the wire should be not URL-encoded. Instead, base64 or whatever encoding should happen here before injecting into JS.

[WofWca]

FYI we do the same (base64) in desktop, for such untrusted content.

[link2xt]

I looked into the payload sent in a place_call_info, and it is an SDP offer, but unnecessarily URL-encoded. Should be a plain SDP instead, there is no need to send all the % over the wire. Otherwise we are stuck with URL-encoded payload for compatibility.

Fixing this needs a fix chatmail/core#7191 in the core first, because it turned out newlines were not tested and break the header.