The Coding Artist

Now Corona is raging the world, the performing arts have a problem. Theaters are closed down and the need for an online performing platform is growing. Of course, you can use Zoom or another ready-made video conferencing platform, but then you are really tight to an interface you might not like or is not suitable for a performance.

Together with Keez Duyves I have build a proof of concept for using jitsi at PIPS:lab. The difficulties we encountered inspired me to write this blog.

Prerequisites:

• Intermediate javascript knowledge
• basic Vue knowledge
• Vue cli should be installed globally on your computer
• npm should be installed on your computer

All the code can be found on the github repository.

Step 0: First Vue app

I will use the Vue cli to create a very basic Vue app from which we can work. I am not interested in styling right now, so I will only focus on the plain html.

In order to create a Vue app proceed as follows in the terminal:

vue create jitsi-vue

Choose the default options to generate the first Vue app. Now go to the directory and run:

npm run serve

Now you will see the following default welcome page in the browser:

The code so far can be found here

Step 1: Empty component

Of course, we don't want this, so let's delete the HelloWorld component and change the App.vue file like this:

<template>
    <div id="app">
        <h1>Performance</h1>
    </div>
</template>

<script>
export default {
    name: 'App',
}
</script>

<style>
#app {
    font-family: Avenir, Helvetica, Arial, sans-serif;
    -webkit-font-smoothing: antialiased;
    -moz-osx-font-smoothing: grayscale;
    text-align: center;
    color: #2c3e50;
    margin-top: 60px;
}
</style>

Now we are ready to start with the jitsi integration.

The code so far can be found here

Step 2: Jitsi library and connect

We are going to use the low level jitsi meet api, so we have full control over the look and feel. The downside is that we have to create all the functionality ourselves but that is exactly what we want.

First we need the library, which we will include using npm:

npm install @lyno/lib-jitsi-meet

Include it in the Vue file using an import statement:

import JitsiMeetJS from '@lyno/lib-jitsi-meet';

Let's try to follow the getting started from the jitsi documentation. We will first perform all the steps in the component itself and then refactor if needed. First get things working, than get things right.

So step one should be the initialisation of the library:

JitsiMeetJS.init();

In order to check something actually happened, we can have a look at the console of the browser and check if anything is logged there::

Success! Something definitely happened there. Now we are ready for the next step:

var connection = new JitsiMeetJS.JitsiConnection(null, null, options);

So there is the first problem: what should the options be? For now, we use an empty object (and we change the var into a const in order to adhere to more moderns javascript standards.)

const options = {};
const connection = new JitsiMeetJS.JitsiConnection(null, null, options);

Then the getting started guide informs us that we can add some listeners to the connection. For now we will simply log the arguments by which the listeners are called to the console:

function onConnectionSuccess() {
  console.log("onConnectionSuccess", arguments);
}
function onConnectionFailed() {
  console.log("onConnectionFailed", arguments);
}
function disconnect() {
  console.log("disconnect", arguments);
}

connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_ESTABLISHED, onConnectionSuccess);
connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_FAILED, onConnectionFailed);
connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_DISCONNECTED, disconnect);

When we run this in the browser we get an error in the console:

Uncaught ReferenceError: $ is not defined

That's an unexpected result! There was no mention of dependencies, but apparently we need something of a $ function. First guess of course should be jQuery, so let's add that to our code. First by getting it using npm and then including it in our App.vue file.

npm install jquery

In the App.vue file:

    import $ from 'jquery';
    window.$ = $;

When we reload the 'run serve' again we are one step further and get the following error:

Hmm. What to do? Clearly we need hosts settings. The documentation states (although not in the getting started, at the bottom) we have to provide the following options:

1. serviceUrl - XMPP service URL. For example 'wss://server.com/xmpp-websocket' for Websocket or '//server.com/http-bind' for BOSH.
2. bosh - DEPRECATED, use serviceUrl to specify either BOSH or Websocket URL.
3. hosts - JS Object
4. domain
5. muc
6. anonymousdomain
7. useStunTurn -
8. enableLipSync - (optional) boolean property which enables the lipsync feature. Currently works only in Chrome and is disabled by default.

We are going to use the publicly available server located at https://meet.jit.si/ and after some peeking around in other projects it turns out that on every jitsi server there is a publicly available config.js file that includes these options and many more. For the public server it located at https://meet.jit.si/config.js. Let's download it and include it in our code. In order to do that, create a new folder in the src directory called options and copy the config file into this directory. The directory structure will look like this now:

In order to import the config in our file we have to transform it to a es6 module by replacing

var config = {

by

export default {

Also, the structure of the file seems to be old as it has no serviceUrl defined. We add that key to the object by taking the 'websocket' value and assign it to 'serviceUrl':

{
    .....
    serviceUrl: 'wss://meet.jit.si/xmpp-websocket',
    .....
}

Now we can import the config as options and feet it into the connection initializer:

import options from './options/config';

Which is in place of the previous definition of options. The complete App.vue script portion will look something like this:

import JitsiMeetJS from '@lyno/lib-jitsi-meet';
import $ from 'jquery';
window.$ = $;

import options from './options/config';

JitsiMeetJS.init();

const connection = new JitsiMeetJS.JitsiConnection(null, null, options);

function onConnectionSuccess() {
  console.log("onConnectionSuccess", arguments);
}
function onConnectionFailed() {
  console.log("onConnectionFailed", arguments);
}
function disconnect() {
  console.log("disconnect", arguments);
}

connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_ESTABLISHED, onConnectionSuccess);
connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_FAILED, onConnectionFailed);
connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_DISCONNECTED, disconnect);

connection.connect();

export default {
  name: 'App',
}

Now Vue in the default configuration might throw error messages due to linting problems in the config file. You can than fix the linting problems in the file. After having fixed the errors and looking into the log, you will still see two errors (although it does a lot more already!) which have to do with CORS:

This seems to be related to the 'externalConnectUrl' and 'bosh' properties. They start without defining the protocol:

'//meet.jit.si/......'

Now on localhost it triggers a redirect because localhost is not secure. This causes a CORS error. If we explicitly ask for https this error goes away:

'https://meet.jit.si/........'

So in the config file you have change all the urls starting with '//' to start with 'https://'.

If you did all this and reload the page, there should be no console errors anymore. You should even see the message 'onConnectionSuccess' somewhere hidden in the many messages the jitsi client sends. In order to clean up the messages a bit, we can set the logLevel using:

JitsiMeetJS.setLogLevel(JitsiMeetJS.logLevels.INFO);

Success! Now we have connected, and we can clearly see it in the console now:

The code so far can be found here

Step 3: Joining a room and creating local tracks

Now we are able to create a connection the next step is to join a room. As the quick start guide states we need to perform the following actions:

room = connection.initJitsiConference("conference1", confOptions);
room.on(JitsiMeetJS.events.conference.TRACK_ADDED, onRemoteTrack);
room.on(JitsiMeetJS.events.codsnference.CONFERENCE_JOINED, onConferenceJoined);

And then:

room.join()

So let's implement it, using console logs for onRemoteTrack and onConferenceJoined. We only can join a room once the connection has been established, so we need to implement the code in the onConnectionSuccess function. For the options we will add an empty object for now:

function onConnectionSuccess() {
  console.log("onConnectionSuccess", arguments);

  const room = connection.initJitsiConference("my-secret-conference", {});
  room.on(JitsiMeetJS.events.conference.TRACK_ADDED, onRemoteTrack);
  room.on(JitsiMeetJS.events.conference.CONFERENCE_JOINED, onConferenceJoined);
  room.join();
}

Now the last thing we have to do is to create the local tracks:

JitsiMeetJS.createLocalTracks().then(onLocalTracks);

Again we don't exactly know what the callback onLocalTracks should look like, so we will write the arguments to the console. The code to create the tracks will now look like this:

function onLocalTracks() {
  console.log("onLocalTracks", arguments);
}

function onConnectionSuccess() {
  console.log("onConnectionSuccess", arguments);
  const room = connection.initJitsiConference("my-secret-conference", {});
  room.on(JitsiMeetJS.events.conference.TRACK_ADDED, onRemoteTrack);
  room.on(JitsiMeetJS.events.conference.CONFERENCE_JOINED, onConferenceJoined);
  room.join();

  JitsiMeetJS.createLocalTracks().then(onLocalTracks).catch(error => {
        console.error("There was an error creating the local tracks:", error);
      }
  );
}

When we run the code now, we get an error again:

Apparently we need to add some options for the createLocalTrack function to succeed. From peeking around in other code, we at least need the devices option so let's add it:

  JitsiMeetJS.createLocalTracks({
    devices: ['video', 'audio']
  }).then(onLocalTracks).catch(error => {
        console.error("There was an error creating the local tracks:", error);
      }
  );

And if we now reload the page we have success!

It turns out we get the created tracks as an array containing one video and one audio track.

Now we have the tracks we should see ourselves in the public room on the jitsi meet server. I called the room 'my-secret-conference' so let's go to the room url and join on https://meet.jit.si/my-secret-conference.

Surprise!! Although we have tracks and we have confirmation that we have joined the room, nothing can be seen there :-(

After looking into some other implementation it seems that the local tracks need to be added to the newly created room. So we will create the localTracks once the conference has been joined. If we inline that function we can reach the room variable from within the closure. So the code will be:

function onConnectionSuccess() {
  console.log("onConnectionSuccess", arguments);
  const room = connection.initJitsiConference("my-secret-conference", {});
  room.on(JitsiMeetJS.events.conference.TRACK_ADDED, onRemoteTrack);
  room.on(JitsiMeetJS.events.conference.CONFERENCE_JOINED, () => {
    console.log("onConferenceJoined", arguments);
    JitsiMeetJS.createLocalTracks({
      devices: ['video', 'audio']
    }).then((tracks) => {
      console.log("onLocalTracks", tracks);
      tracks.forEach(track => {
        room.addTrack(track);
      })
    }).catch(error => {
          console.error("There was an error creating the local tracks:", error);
        }
    );
  });
  room.join();
}

We have now successfully joined a room, and we have two tracks. But the javascript is not in the Vue component, and it's plain ugly!. We will refactor the code in the next section.

The code so far can be found here

Step 4: refactoring the rather ugly code

So far we have been trying to get the things working in the simplest way. Now it's time to clean up the code. First thing we notice is that there is a callback hell going on. Lets cleanup that by using promises. First the connection code:

function connect() {
  return new Promise(((resolve, reject) => {
    const connection = new JitsiMeetJS.JitsiConnection(null, null, options);
    function onConnectionSuccess() {
      console.log("onConnectionSuccess", arguments);
      const room = connection.initJitsiConference("my-secret-conference", {});
      room.on(JitsiMeetJS.events.conference.TRACK_ADDED, onRemoteTrack);
      room.on(JitsiMeetJS.events.conference.CONFERENCE_JOINED, () => {
        console.log("onConferenceJoined", arguments);
        JitsiMeetJS.createLocalTracks({
          devices: ['video', 'audio']
        }).then((tracks) => {
          console.log("onLocalTracks", tracks);
          tracks.forEach(track => {
            room.addTrack(track);
          })
        }).catch(error => {
              console.error("There was an error creating the local tracks:", error);
            }
        );
      });
      room.join();
      resolve(connection);
    }

    function onConnectionFailed() {
      console.log("onConnectionFailed", arguments);
      reject(arguments);
    }

    function disconnect() {
      console.log("disconnect", arguments);
    }

    connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_ESTABLISHED, onConnectionSuccess);
    connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_FAILED, onConnectionFailed);
    connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_DISCONNECTED, disconnect);

    connection.connect();
  }))
}

As you can see, all the room creation and joining is done in the connection success. But since we have now have a promise that resolves with a connection, we can extract all the room code out. We also can take out some of the console log statements and inline the eventlisteners using arrow functions.

function createAndJoinRoom(connection) {
  const room = connection.initJitsiConference('my-secret-conference', {});
  room.on(JitsiMeetJS.events.conference.TRACK_ADDED, onRemoteTrack);
  room.on(JitsiMeetJS.events.conference.CONFERENCE_JOINED, () => {
    console.log('onConferenceJoined', arguments);
    JitsiMeetJS.createLocalTracks({
      devices: ['video', 'audio']
    }).then((tracks) => {
      console.log('onLocalTracks', tracks);
      tracks.forEach(track => {
        room.addTrack(track);
      });
    }).catch(error => {
          console.error('There was an error creating the local tracks:', error);
        }
    );
  });
  room.join();
}

function connect() {
  return new Promise(((resolve, reject) => {
    const connection = new JitsiMeetJS.JitsiConnection(null, null, options);

    connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_ESTABLISHED, () => {
      resolve(connection);
    });
    connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_FAILED, () => {
      reject('The connection failed.');
    });
    connection.addEventListener(JitsiMeetJS.events.connection.CONNECTION_DISCONNECTED, () => {
      console.log("Connection disconnected");
    });

    connection.connect();
  }))
}

connect().then(connection => {
  createAndJoinRoom(connection);
}).catch(error => console.error(error));

The connection code improved a lot! It's much more readable now. The room joining code is still a mess though. Here we will also use promises to make the code more readable and understandable. Furthermore, we will extract the localTracks code. That should be optional and live in its own function. The track added listener will be added later as well, so we can remove that listener as well.

function createTracksAndAddToRoom(room) {
  JitsiMeetJS.createLocalTracks({
    devices: ['video', 'audio']
  }).then((tracks) => {
    tracks.forEach(track => {
      room.addTrack(track);
    });
  }).catch(error => {
        console.error('There was an error creating the local tracks:', error);
      }
  );
}

function createAndJoinRoom(connection, roomName) {
  return new Promise((resolve) => {
    const room = connection.initJitsiConference(roomName, {});
    room.on(JitsiMeetJS.events.conference.CONFERENCE_JOINED, () => {
      resolve(room);
    });
    room.join();
  });
}

Now the calling code will look like this:

connect().then(connection => {
  return createAndJoinRoom(connection, 'my-secret-conference');
})
.then(room => createTracksAndAddToRoom(room))
.catch(error => console.error(error));

Now the code looks a lot more readable. The only thing is that we should move it to its own file and export the three functions. The code is also not in the Vue component. In order to fix that we will add the connection code in the mounted hook, so the code of the Vue component looks like this:

import { connect, createAndJoinRoom, createTracksAndAddToRoom } from './utils/jitsiUtils'

export default {
  name: 'App',

  mounted() {
    connect().then(connection => {
      return createAndJoinRoom(connection, 'my-secret-conference');
    })
    .then(room => createTracksAndAddToRoom(room))
    .catch(error => console.error(error));
  }
}

The code so far can be found here

Step 5: Displaying the video and audio

Till now, we don't see anything, so the code is rather useless from an end user experience. We need to display the video and audio, so we can actually have a conference. For that to happen we need the tracks in our view component first, so let's create listener which will add the tracks to a Vue data object:

export default {
  name: 'App',

  data() {
    return {
      tracks: [],
    }
  },

  methods: {
    addTrack(track) {
      this.tracks.push(track);
    }
  },

  mounted() {
    connect().then(connection => {
      return createAndJoinRoom(connection, 'my-secret-conference');
    })
    .then(room => {
      room.on(JitsiMeetJS.events.conference.TRACK_ADDED, track => this.addTrack(track));
      createTracksAndAddToRoom(room);
    })
    .catch(error => console.error(error));
  }
}

Now we have the tracks in our component we can inspect the data very easily in the Vue inspector: We have two tracks: the video- and the audio track. We want to separate them because the video track needs a video element and the audio track needs an audio element:

  data() {
    return {
      videoTracks: [],
      audioTracks: []
    }
  },

  methods: {
    addTrack(track) {
      if (track.getType() === 'video') {
        this.videoTracks.push(track);
      } else if (track.getType() === 'audio') {
        this.audioTracks.push(track);
      }
    }
  },

The only thing to do now is to render the tags and once rendered attach the tracks to the elements.

<template>
  <div id="app">
    <h1>Performance</h1>
    <video v-for="track in videoTracks" :key="`track-${track.getId()}`" :ref="track.getId()" autoplay />
    <audio v-for="track in audioTracks" :key="`track-${track.getId()}`" :ref="track.getId()" autoplay />
  </div>
</template>

  methods: {
    addTrack(track) {
      if (track.getType() === 'video') {
        this.videoTracks.push(track);
      } else if (track.getType() === 'audio') {
        this.audioTracks.push(track);
      }
      this.$nextTick().then(() => {
        track.attach(this.$refs[track.getId()][0]);
      })
    }
  },

Notice the attaching of the track in nextTick: we have to wait until the element is in the dom. Also there is something strange with refs in v-for loops. The refs are arrays, so we need the first element in the array to get the real ref.

Success again! We are able to render both the video and audio using the jitsi-meet library.

The code so far can be found here

Step 6: Houston, we have a problem!

Now we seem to have a working solution. We are able to connect to a room and display our own video. Let's open a second window and check if we really have a connection.

If we do that there is no success. Somehow, though the jitsi-meet library says that we are connected, we clearly aren't! Browsing the jitsi documentation is, as we saw earlier, of no help at all. Peeking again in other code that uses this library teaches us that we need to add the room name to the serviceUrl using a url parameter. We can achieve that by adding the room name as a parameter to the connect function and create a custom options object using the room name:

export function connect(roomName) {
  return new Promise(((resolve, reject) => {
     let optionsWithRoom = { ...options };

     optionsWithRoom.serviceUrl = options.serviceUrl + `?room=${roomName}`;
     const connection = new JitsiMeetJS.JitsiConnection(null, null, optionsWithRoom);

    ...
  }))
}

Another problem might be that the browser wants to see some user activity before creating the tracks. So lets move the connection code from the mounted hook to the methods, so we can connect on a button press:

  methods: {
    ...
   connect() {
      const roomName = 'my-secret-conference';
      connect(roomName).then(connection => {
         return createAndJoinRoom(connection, roomName);
      })
      .then(room => {
         room.on(JitsiMeetJS.events.conference.TRACK_ADDED, track => this.addTrack(track));
         createTracksAndAddToRoom(room);
      })
      .catch(error => console.error(error));
   }
},

We need a button to be able to connect now, so let's add that in the template:

<template>
  <div id="app">
    <h1>Performance</h1>
    <button @click="connect">Connect</button>
    <video v-for="track in videoTracks" :key="`track-${track.getId()}`" :ref="track.getId()" autoplay />
    <audio v-for="track in audioTracks" :key="`track-${track.getId()}`" :ref="track.getId()" autoplay />
  </div>
</template>

If we test with two windows now, we see that there is really a connection!

The code so far can be found here

Conclusion

I showed the process I used to implement an unknown library. We have a very rudimentary version working but there is still a lot to be done:

1. Connect and disconnect buttons instead on connecting at enter
2. Muting or not rendering the audio component for the local audio track
3. Listening to other events (like track removal!)
4. Separating the video and audio elements in separate Vue components so the mounting is abstracted away
5. Add styling
6. Many, many more things to do

I leave this to you. From now on you can use the jitsi documentation and your Vue knowledge to create the rest.

Update January 23, 2021

I added step 6 as without it the code didn't seem to work. I also updated the version of the library in the code that is on github.