Observation and Proposal


1. Observation

In order to observe two user groups (presenters and participants), we watched different recorded Zoom presentations and generated sticky notes about our findings on white board software Miro to define mental models. In the same videos, we observed the presenters and participants at the same time. To validate our findings, we often revisited a particular scene and watched in pairs.

After listing all distinguished tasks on the white boards, we grouped them by behavior affinity.

Mental model 1 Mental model 2

Presenters struggle to solicit and assess participant engagement and responses to presentation. Presenters have been shown to ask (or sometimes appear to plead in the instances of educational presentations) for participant feedback. Participants that are familiar with the presentation do manage to provide feedback through unmuting and speaking, messaging in chat, or using features like hand raising. Unfortunately, most participants don’t provide any feedback to the presenter and have their camera off, seemingly disengaged. This appears to make presenters feel uncomfortable. On the other hand, when presenters want to assess participant reaction (for instance by reading the chat), it often interrupts their presentation.


Presenters

As far as presenters are concerned, they can be grouped by two core attributes:
prior knowledge of video conferencing and degree in interest to engage people.

Mental model 1 Mental model 1

Participants

Then, we grouped participants into different categories based on two core attributes: participation and focus

Mental model 1 Mental model 1


In order to keep it manageable within the allotted time frame, we selected core groups to focus for this project. For presenters, we decided to target those with low and medium prior knowledge of video-conferencing tools , as well as those who look forhigh participation from their audience. For participants, we targeted both shy and active participants that are focused on the presentation.


2. Problem identified



The inability for presenters and participants to engage and have dialogue with one another is a major issue with video conferencing. The lack of low risk, non-disruptive, simple-to-use features that allow participants to comfortably engage with presentations results in lower levels of engagement. For many presenters, participant engagement and reaction serves as a measure of the presentation’s success. Unfortunately, because of the suppressed participant reaction, presenters can be made to feel boring and awkward. Furthermore, this depression of participation can make participating even harder as the norm becomes silence and turned off webcams, making speaking up or asking a question all the more intimidating and daunting to participants. This lack of engagement is especially troublesome for presentations with educational objectives as engagement is a necessary component for learning. So far, we have yet to find a product that finds a way to support participants' engagement and interaction with the presentation.




3. Personas

...
  • Diane
  • Presenter in informal education community
  • Age: 30s
  • Role: presenter
  • Prior knowledge of video conferencing software: some experience
  • Attitude towards participation: Cares about and solicits interaction
  • Presence on video conferencing tools: Zoom, Skype and facebook

Diane is a young professional who is a member of a peer supported informal online learning community. A recurring activity in her community is a peer-teaching event where different members of the community present on a topic of their choosing. Being a PhD student, Diane is familiar with teaching and giving presentations as she has worked as TA for several years. She recently started giving presentations online due to the pandemic but finds presenting online is just not the same. Considering video conferencing software, she has used Zoom, Skype and facebook video call. Trained to read body language and facial expressions, Diane feels like she has no sense of how her participants are doing through an online setting. Despite her requests for participants to tell her how they’re following along, participants often have their cameras off and only the same few people interact with her. Furthermore, every time she tries some interactive features like polling or emoji reactions, she struggles to set up them up and visualize participant responses, this makes her lose time and slows down her presentation. This is a serious problem for her and interferes with her teaching ability and satisfaction with her work.

...
  • Dakota
  • Student presenter
  • Age: 22
  • Role: presenter
  • Prior knowledge of video conferencing software: Tech savvy
  • Attitude towards participation: Cares about and solicits interaction
  • Presence on video conferencing tools: Zoom, Skype, Google hangout, and Duo

Dakota is a fourth-year university student. They are actively looking for job opportunities in business development. Dakota is a natural leader and is always willing to be a presenter for the team project. They know how to interact with classmates and joke around to make their presentation more enjoyable. They also enjoy seeing their peers’ reactions to their jokes and comments. As video conferencing seems to be the new normal, they want to be more familiar with tools to give a presentation that amaze everyone. Dakota often imagines themself sitting relaxed, smoothly interacting with participants and making them more engaged in the presentation. They have tried many video conferencing tools such as Zoom, Skype, Google hangout and Duo but are unsatisfied with them.

...

  • Jamie
  • Business convention participant
  • Age: mid-50s
  • Role: Participant
  • Attitude towards participation: Active participants
  • Degree of focus: Highly focused
  • Presence on video conferencing tools: Zoom, Facebook, and Google hangout.

Jamie is a long time member of a well respected business convention. Jamie has been attending this convention for decades and he is well known in this community. The convention Jamie attends has moved online recently and it has changed the way Jamie feels about it. Before, Jamie loved to see his peers, watch their reactions, and participate in lively discussions during business presentations. Jamie has no idea how the virtual conference works at this time. Although he uses facebook video calls to chat with his children living abroad, he has never used video conferencing tools for meetings. Jamie is still looking for that lively engagement and discussion during presentations. He tries out as many features as possible to express his reactions without distracting others during the presentation, but is disappointed when he finds out that he is the only one participating in this way, or at all.

...
  • Priscillia
  • University student
  • Age: 19
  • Role: Participant
  • Attitude towards participation: Shy participants
  • Degree of focus: Highly focused
  • Presence on video conferencing tools: Zoom, facebook, Jitsi, google hangout and skype.

Priscillia is a first-year university student. She was so excited to meet new friends and learn the topic she is interested in at University, then the pandemic happened and all classes were moved to online. She has never had an online class before but has watched some videos from Cousera and Khan academy while studying in CEGEP. During the online class on Zoom, she is often hesitant to jump in the discussion and speak as she doesn’t know anyone in a class yet. She sometimes uses the raise hand feature but it is often not noticed. Everything is so new and she feels totally unprepared and overwhelmed. Most importantly, she usually enjoys actively participating in the class and answering questions, but feels way too afraid to be put on the spot which prevents her from participating.


4. Use cases

Presenter use case
  • In this use case scenario, Diane wants to host an online educational presentation for her community.
  • She opens the video conferencing app, logs in using her credentials, and starts an online meeting by clicking the “Start Meeting” button.
  • She is presented with a link to the meeting, share it with her audience, and waits for people to join.
  • As the participants trickle in, she uses this time to get to know her peers by asking some questions, like: “Are you familiar with the presentation topic? Raise your hand if yes”
  • Now that all the participants have arrived, she starts a quick poll about the topic they are covering where participants vote using A/B/C/D. She is able to quickly assess their response as the results live-tally on screen and reacts accordingly.
  • During the more technical phase of the presentation, she notes that a lot of participants have anonymously reported they are beginning to feel lost. This cues her to slow down and cover the material with more details so participants don’t feel lost.
  • At the end of the presentation, the participants overwhelmingly give her clapping emojis and she feels satisfied knowing she did a good job!

Participant use case
  • In this use case scenario, Priscillia is attending one of her online classes.
  • She follows the link given by the instructor to attend a class and joins the class early.
  • The class already has quite a few students in it, so she doesn’t feel comfortable using their microphone to say ‘Hi’ to the others. Instead, she gives a “wave” emoji to signal her presence.
  • As the class is starting, the teacher asks the students to tell them how they feel with an anonymous emoji reaction. Priscillia knows she is anonymous so she answers honestly with a sleepy face emoji. She watches as the live tally of student's emoji reactions pour in: many students are responding with a “meh”, “sleepy” or x_x face emoji. She feels a little less alone.
  • The teacher asks review questions covering content in the previous class. Again, because the poll is anonymous and the button to respond is on the main app interface, she responds.
  • During the discussion of new content, Priscillia begins to feel lost. She emotes anonymously that she feels lost and sees the tally of lost feeling students increase by one. Soon the number begins to climb to about ÂŒ of the class, so the teacher pauses their instruction to solicit clarifying questions from students.
  • Students begin raising their hand and a queue is populated based on first come first serve. Priscillia was 3rd to raise her hand so she waits for her turn.
  • It turns out that the first question asked is the same as hers, so she lowers her hand which automatically removes her from the queue of questions.
  • By the end of the class, the professor again asks students to anonymously respond with how well they understood that day's content with an emoji poll. She responds with a green checkmark.



5. Related products


Zoom

Due to its high level of security and variety of useful features that support educators, Zoom is used as a virtual classroom platform in many educational institutions including McGill University. On Zoom, students are able to provide feedback to their lecturers via emoji and express their reactions by sending a clap or thumbs up. Moreover, students can display a “raise hand” icon if they want to get permission to speak during the class. These various features of Zoom allow students to transform their behavior in off-line class into a virtual class so that students are able to feel strong telepresence throughout the class.

However, these emoji buttons are scattered throughout the software which sometimes results in undesired confusion. For example, you can express “thumbs up” via emoji reactions tab or participants list. Users will likely be confused with the differences between the two. In addition, emojis reactions are often not noticed by lecturers as features like screen sharing often occupies the entire lecturer’s screen. Therefore, students often don’t get a chance to interact with an educator.

Finally, the current layout of Zoom is poorly designed. Students need to manually place windows of participants and chat on their screen when needed as Zoom only provides one default layout; chatbox on the left corner, participants list on the left top and screen shared on the middle. Therefore, on a default layout if a chat box is too small due to one’s laptop size, having a discussion in a chat is somewhat challenging.


Cisco Webex Training

Cisco Systems also has a video conferencing tool: WebEx, which offers a range of useful features for presenters and participants in online training sessions such as presentations, webinars or online classes.

Cisco Webex Training offers an easy way of tracking questions and documenting responses using a threaded Q&A window as well as collecting feedback from the participants through live polling. Participants are able to react to the presentation through the ‘raise hand’, ‘yes’, ‘no’, ‘go faster’ and ‘go slower’ buttons as well as express their feelings through emojis.

Mental model 2
Retrieved on September 20 from the Cisco WebEx Training Center User Guide (link)


Furthermore, presenters can track participant’s attention by using the Attention Indicator which shows if a participant has minimized the training session window or has brought another window in focus on top of the training session window. This is useful for recording student’s participation when teaching remotely.

Mental model 2
Retrieved on September 20 from the Cisco WebEx Training Center User Guide (link)


However, WebEx is slow when connecting to meetings and its video quality is lower than Zoom’s. It also does not have a lot of room for customization and only allows to visualize a summary of participant’s yes/no reactions. When reacting with emojis or other buttons, the presenter has to check the symbols next to each participant’s name in the list, which is hard to do in a session with many participants. Additionally, participants can’t react anonymously and WebEx isn’t free to use.


Adobe Connect for learning

Adobe has several video conferencing products for online meetings, online learning and webinars; Adobe Connect is especially designed for educational settings. Adobe Connect has a variety of interactive features both for educators and students.

Mental model 2
Retrieved September 20, 2020 from Adobe Connect for Learning. (link)


First, presenters can easily run a poll including yes or no, multiple choice or open-ended short answers questions during the presentation. The results are visualized right after the poll is closed. Students (or meeting participants) can display reactions via emoji. Apart from “raise hand”, “agree or disagree”, “applaud”, “laugh” and “step away” emojis, they also have “request” emojis for asking educators to “speak louder”, “speak softer”, “speed up” and “speed down”.

Second, Adobe Connect has a very sophisticated feature to assess audience participation. Connect collects and analyses various parameters related to participant engagement and provides presenters with a real-time aggregate “engagement score”. The Engagement Index Calculation uses: chat activity, Q&A activity, poll activity, file download, status updates, and other metrics to come up with the engagement score.

Mental model 2
Retrieved September 20, 2020 from Adobe Connect for Learning. (link)


Lastly, Adobe Connect provides a different layout depending on the purpose of sessions (sharing, discussion, collaboration and customized layout). With a single click, Adobe Connect’s layout is automatically adjusted.

However, Adobe Connect is one of the most expensive available solutions for web conferencing. For example one of their published plans costs 50$ per month and can only host up to 25 participants; for bigger meetings (100-1500 participants) the plans start at 130$ per month.

Furthermore, with the ‘Status View’, hosts can only see the reactions of participants that are raising their hands, agreeing, disagreeing, or have stepped away. For the remaining reactions and requests, the host has to switch to the ‘Participant List View’ and see the reactions next to each participant name (similar to Zoom). Additionally, participants can’t share their reactions anonimously.

Mental model 2
Retrieved September 20, 2020 from Adobe Connect for Learning. (link)


Other disadvantages of Adobe Connect include its dependency on internet bandwidth (when viewing on the browser) or its slowness to load (when using the desktop app). Therewith, the interface might be fairly complicated for new users.


6. Product comparison



One of the main advantages of our proposed system is the ability for participants to react and engage in a presentation without garnering overwhelming attention and/or being disrupting, while still being easily visible to the presenter.

Presenters will be able to interpret participant engagement with ease through our proposed real time visualization and aggregation of participant’s reactions and responses This eliminates the need for presenters to switch to different tabs or change the view mode to engage more comfortably with participants.

Finally, our system is intended to be free and open-source so presenters can customize it to their needs and can use it with any pre-existing video-conferencing tool without having to spend the extra dollars.

The system will benefit our personas in the following ways:

Diane

As a member of a peer-teaching community, our system will allow her to easily visualize her peer’s reactions when presenting a topic during one of their teaching sessions. She will be able to get a sense of how many participants are following and assess their reactions easily. This way, she can immediately pause and give further explanations when the crowd is feeling lost or speed up her presentation if they’re feeling bored. She will be able to visualize all this information on the same window preventing her from losing time and/or participant’s attention during interactive exercises.

Dakota

Dakota will be able to interact with their audience with more ease using our software. They will be able to see when participants found their jokes funny, and if they are liking the content they are presenting. They will also be able to poll their audience to make sure they are following their presentation and realize interactive exercises that engage everyone with the content.

Jamie

With our system, Jamie can actively participate in discussions around business presentations, he will be able to raise his hand without having to dig through several menus or drop downs. He won’t have to interrupt the presentation in order to ask questions and will get a chance to be designated to speak in an orderly manner. Furthermore, he will be able to see other participants' reactions and not feel like he is alone in the presentation.

Priscilla

Finally, Priscilla, the shy student who is afraid of sounding stupid when participating in discussions, and is too shy to giving a wrong answer or asking for clarification when she is lost, will benefit from our system. The ability to anonymously vote in quick polls as well as expressing her feelings and reactions to the presentation without being the center of attention will allow her to participate in low risk ways with the class.


7. High-level design of system


Mental model 2