Neuromatch Academy: The Story

Many people ask me what Neuromatch Academy (NMA) was about, how it worked, how we got there, what we learned etc. And every time I try to provide a concise answer, it feels like I’m not doing it justice. Where to start? What to say? How can one summarize what has been the biggest interactive online neuroscience training event in the history of neuroscience in a few sentences?

It also turns out different people are more or less interested in hearing different parts of the story… Who would have guessed that what’s boring to some is entertaining and interesting to others? But many aspects of NMA are tightly intertwined, only make sense knowing the history of events, and it’s hard to extract specific aspects without providing the broader context.

So this is my attempt at telling the interested reader the story from my perspective. How it started, what we built, how we achieved it, and lessons (I think) we learned. This is the unofficial backstory, a personal account on how I perceived things. It’s not meant to be exhaustive. It’s a story as much as it’s a reflection on what worked and what didn’t. I’m writing this as much for you, the reader, as I’m writing this for myself to organize my ideas and thoughts, memories and goals.

And before I start, I need to thank all of the members of the amazing Neuromatch Academy team. As I will describe later, everyone has played a crucial role, regardless if they were executive leaders or a volunteer implementing our vision. Thank you! Without all and everyone of you, this adventure would not have been possible. I would also like to particularly thank my fellow members of the board of directors. Dear friends, I’m very fortunate to have you in my life! And I am very much looking forward to furthering our overall mission together, to democratize access to high-quality education world-wide.

Finally I need to thank my amazing wife, Dr. Aarlenne Z Khan for her unconditional support when I spent many hours building NMA…

Enjoy!

1. The spark: how everything started…

Computational Neuroscience is a field that is often considered “in support” of experimental neuroscience. Every Neuroscience department will have a few modelers, but this hardly ever reaches a critical mass needed for high-quality computational training across all areas of neuroscience. In addition, many institutions do not have modeling experts in their neuroscience teams. As a result, access to high-quality computational neuroscience training within institutions is rare. This is why summer schools in computational neuroscience have been a crucial driver of high-quality education in the field, for a long time .

The impact of summer schools like the Cold Spring Harbour Vision school or the Okinawa school has been undeniable. Graduates from those schools often still use the professional network they’re now part of. Long hours of work in small groups forge life-long connections leading to collaborations and friendship. And in certain areas, having participated in the leading summer school of the field means tremendously increased chances for securing a faculty position at a first class research institution. Summer schools thus provide much more than specialized training; they provide unmatched opportunities for scientific networking and career advancement.

Realizing the need for a summer school that provided such scientific networking and high-quality training in computational systems, behavioral and cognitive neuroscience, I created the in-person 2-week long Computational Sensory-Motor Neuroscience (CoSMo) summer school in 2011 and ran this school together with Konrad Körding and Paul Schrater for 8 consecutive years (see past open access content here). We paused CoSMo in 2019 to overhaul the curriculum and fully integrate modern machine learning approaches with classical computational neuroscience techniques. Paul, Konrad and I got together with our families at a beautiful cottage near Calabogie, ON for a 2-week retreat to envision the new teaching materials. This new CoSMo summer school was supposed to run in person during the summer of 2020 and we had just recruited Megan Peters (CoSMo 2013 alum) to help us with that. We had funding from the Canadian Institute for Advanced Research (CIFAR) as well as from Konrad’s National Science Foundation (NSF) grant all lined up and we were about to open applications in March 2020. But then things rapidly changed.

In March 2020 the Covid-19 pandemic hit the western hemisphere and conferences and summer schools world-wide had to be cancelled. Within days of lockdown it became clear that there was not going to be a quick recovery and return to normal, despite many overly optimistic voices. We were really excited to launch the new CoSMo program to bring the integrated power of machine learning and traditional neuroscience approaches to our research communities. We were (and still are) certain that the smart integration of these approaches would lead to new, accelerated scientific breakthroughs. But like everyone else, on March 22, 2020 we decided that we had to cancel CoSMo 2020. And given the general paralysis of the early pandemic days, in a way this was a relief; one less thing on the to do list while dealing with homeschooling, much reduced work hours, all kinds of contingency planning for our labs and lives, etc. We were ready to just put our heads in the sand. But then it came otherwise…

Email from Konrad in my inbox on April 3, 2020: “what about hosting a [online] CoSMo where we have every day someone doing teaching and tutorials? I think there may be a lot of need for such a thing. We could run it for free. On an unlimited audience.” This was the spark that ignited Neuromatch Academy! Four days and about 100 emails later, we were sure that organizing an online summer school was crucially necessary: hundreds of students, a whole generation of young talents would otherwise miss out on quality training in computational neuroscience. So we devised our first plan: contact all other computational neuroscience summer school organizers worldwide and see if they’d be interested to team up. After all, they all had to cancel their in-person programs too! We created a Slack channel to avoid the already overwhelming amounts of emails. And we had a rough time horizon: July 2020 would be when this online summer school would happen. We had 3 months to prepare for that. No problem, all we had to do was put a bunch of recorded lectures and tutorials online. Done. Easy. “It won’t be much work…” (Konrad)

2. Building up the fire: the birth of NMA

So we decided to create an online summer school. But would anyone really be interested in this? After all, we were in at the start of a pandemic, there was a high degree of uncertainty regarding time commitments, and maybe the idea of an online summer school would just turn people off? Would people really want to sit in front of a computer screen at home all day trying to watch lectures and do tutorials? We were hoping that maybe a couple hundred people would take part. We’d recruit a few teaching assistants to help us out with questions and help students work through the tutorials. But was there any interest at all? We needed to know. So we (i.e. Konrad) set up a simple Twitter poll. The response was overwhelming! Several thousand people indicated interest.

Now throwing a bunch of lectures and tutorials online that we hack together the night before teaching them (as often the case during in-person summer schools) might work for a couple hundred people. When numbers are small it’s easy to correct small errors and improvise on the fly. But several thousand people? There was no way we could handle error messages and help requests from thousands. Imagine your inbox exploding. Imagine the frustration of online participants. People would have just given up and quit. This would have been a disaster.

We quickly realized that catering to thousands instead of tens or a few hundred would be another beast altogether. Our materials would need to be highly curated and tested in advance so that they are guaranteed error free. We needed to make sure they’re clear, understandable, doable in the allotted time, and complete. We also realized that this would not just be a random local summer school, but a world-wide learning academy. We realized that time zones would present a huge challenge in teaching a summer school online and including everyone world-wide. We realized that building this would take much more than our little 6-person team (we had recruited Sean Escola and Brad Wyble to the team already at this point). And this is when the Neuromatch Academy frenzy started. But more about that later.

There was another important thing we realized: we better figure out what people’s expectations are. What do potential attendees actually want to learn? Which techniques and approaches are considered crucial for computational neuroscience? And once again, we crowdsourced this information. We asked people on Twitter to list all the topics they thought were fundamental to computational neuroscience. And we then used these together with expert knowledge from our team to devise a coherent curriculum. More on that later too.

But the hardest question of all was the following: summer schools build communities. There is a real benefit coming from the shared “suffering” of often several weeks of intense, long days of lectures, tutorials, projects and other activities, chronic lack of sleep of both attendees and instructors (because everyone worked late and then decompressed socially). All this creates life-long memories and bounds between participants (and instructors) that are often career defining. Many collaborations have arisen from summer schools. Papers come out of group project ideas. The scientific network promotes career advancement and mentoring. Access to the field’s leaders teaching at summer schools is a privilege and huge benefit to participants. How in the world can we recreate such an environment in an online setting catering to thousands instead of just a select few dozen?

3. Growth — let the frenzy begin

All those questions required fast answers. After all, we only had 3 months to develop a coherent curriculum, test it, curate it and deliver it. Though at the beginning of NMA, we did not see the huge amount of work this would quickly become, we realized very quickly that we would need a lot of help. We needed people with summer school experience and expertise in teaching. People who were broadly educated in computational neuroscience for curriculum design. People who would help us advertise, select students and recruit TAs. People that would be good at creating didactic tutorials. Great lecturers. We would also need administrators. By the end of April 2020, we had 50 active NMA members in Slack. And this was just the start.

When you want to have an online event of a sort that has never been attempted, you better make sure that you have someone on board who knows the technical side of things. So one of the most crucial volunteers would be a specialist in information technology to help make decisions on platform and technology choices, and eventually to help with content curation. We got super lucky to find Patrick Mineault. Patrick turned out to be not just a critical and reliable helper with IT-related questions. Patrick also came with corporate experience in big project management. It turned out, as academics, we all were pretty much clueless about proper project planning and execution. Who would have known that such skills are actually useful…? But all jokes aside: putting together a detailed timeline for rolling out all aspects of NMA was one of the most eye-opening and crucial contributions that Patrick made. Ultimately, he took us from a likely failure to a likely success. And thanks to him, we did succeed (read more about Patrick’s contribution here).

However, we were still a bunch of amateurs scrambling to make things happen. We in the board of directors were essentially the main drivers of the enterprise. We led NMA on all levels, leading committees, being involved in everything and anything. There was little structure in terms of human resources. We were chaotic, frenzic, and over-enthusiastic troupers eager to make NMA happen. We all had our fingers in everything and at times that became quite overwhelming. Especially just before rolling out NMA when our Slack communication peaked at around 5,000 messages each day! But it was all for a good cause, so who’s counting, right?

Despite all the productive chaos, or maybe thanks to it, we realized we did need some structure. So we created different working groups and committees to tackle different questions and organizational aspects. We had a curriculum committee, a group project committee, a diversity committee, etc. And each committee had a chair who was part of our newly created executive committee. It was early May 2020 and our organization took shape. But we also kept growing. By the end of June 2020 we had around 250 active Slack members helping us with every aspect of the NMA organization and materials development. Different working groups, committees, sub-committees, and fire fighting teams seemed to be dynamically emerging and self-organizing from the chaos all the time.

This organic way of self-organization worked fairly well for us most of the time. It was great that things just happened because so many people were excited and motivated to join us and help with all and every aspect of NMA. There was one aspect that we realized early on would not just magically work without a real structure: and that was finances. We had no means to receive funding, collect participant fees or pay TAs and other services (like accounting). We were not an official organization, nor were we at a single institution that could funnel finances on our behalf. At first we explored piggy-bagging onto existing organizations. We talked to several conference organizers as well as existing non-profits in the field. All wanted to help us, but in the end all lacked flexibility, took too high of a cut and/or would be too cumbersome or slow to work with. We were going at max speed and we couldn’t afford to get slowed down.

This is when we decided we needed to create our own non-profit organization. This was going to be a critical step towards independence and (hopefully) longevity. But it would involve a lot of paperwork… Luckily, lawyer firms offer pro-bono services to nonprofits. So we applied for this through the American Bar Association. Since 5 out of 6 of the NMA board of directors resided in the USA, creating a US nonprofit was the most logical choice for us. And I must say we got extremely lucky. We managed to secure an amazing lawyer firm. I wish I could tell you who they are but they prefer to stay anonymous. In any case, this team of lawyers helped us file for 501(c)(3) nonprofit status in the US, we became a nonprofit company a month later and were officially tax exempt early August 2020.

Being incorporated was huge for us and keeps paying off. First of all, we could now open bank accounts, receive sponsorship funds, pay bills and — most importantly — pay our TAs for their hard work. But it also meant that we were now a real legal entity with fiscal responsibilities and a duty to serve the community. Not that we needed this encouragement, but it made it official. And it confirmed in a way that we were going to make a permanent change to computational neuroscience education. It also validated our efforts. Now we were official. The “board of directors” had a meaning (the IRS has our contact info). All the amazing volunteers involved in helping us were now helping an official nonprofit. We were on course for success and making an impact in the field.

4. Burning down barriers: NMA’s vision and mission

One really fun thing you get to do when starting something new — especially a nonprofit — is to think about your nonprofit’s vision. How is it that you want to save the world?

Initially, when we started NMA, we just wanted to fill the void left by all the amazing summer schools that had now been canceled due to the global pandemic. But when we got the first expression of interest poll results back and saw that potentially thousands of students from all over world would like to participate, we realized that we were doing something much bigger that we needed to fully embrace: we were making high-quality training in computational neuroscience accessible to anyone, (almost) anywhere in the world.

Up to now, summer schools have been reserved for a small elite of trainees who could afford the typical cost of US$3–5,000 of a summer school. Travel costs alone were a huge barrier to participation. Housing, food and registration fees added to this financial barrier. As a result, only trainees from rich labs (or from a very privileged socioeconomic backgrounds) could afford participation in these traditional in-person summer schools. Equally importantly, there were many other barriers to participation, such as having a family to take care of, suffering from disabilities making travel difficult, health issues, discrimination, and being restricted from traveling to certain countries due to geopolitical nonsense. So it turns out that the in-person summer schools we have been organizing — while amazing training opportunities — were also highly implicitly discriminatory with huge barriers to attending them.

Reflecting on this implicit discrimination of in-person summer schools really affected me personally and opened my eyes. I must admit that while I was somewhat aware of this, I just always thought that there was nothing much we could do but try to lower access barriers. For example, at CoSMo we made sure there was childcare available, that lecture rooms and housing was wheelchair accessible etc. We spent a lot of time and effort fundraising to lower access cost. But there was only so much we could do. While we were able to dramatically reduce the cost of CoSMo — and offer it for free in some years — we were rarely able to help with travel costs, especially for people from non-US or non-Canadian provenance (due to restrictions on the use of external funds). So to all the people who were never able to access CoSMo, my sincere apologies.

What an opportunity to overcome access barriers with the creation of an online school. In creating NMA it quickly became clear that we wanted to provide an inclusive online summer school with all (or as many as possible) benefits of in-person schools but without the barriers. We wanted everyone and anyone with access to the internet to benefit from NMA instruction and materials. And of course we realize that there are still many barriers; internet access is not a given in many places around the world. Being able to forego 3 weeks of salary while participating in a summer school is not a given. Being free of having to take care of loved ones is a privilege that many don’t have. It’s important to us at NMA to continue reflecting and listening to the community to identify barriers and devise potential means to overcome them.

Overcoming the financial barrier was one of our main concerns for NMA 2020. Access would already be democratized, since you only needed a computer with internet access to take part in NMA. But we also wanted to make it cheap for all and free for those who could not afford the small price tag of $100 adjusted to each country’s median income (we suggested payments of $100, $60 or $30 depending on median income; anything below was automatically waived). Each participant could waive that cost through the click of a button, no questions asked. But for that to be possible while still paying our TAs and prepod course testers, we needed to raise money. This is where Sean Escola’s magic came in. Within weeks he fundraised roughly 400k USD from major players in the field, both industries and research foundations. I’m not qualified to explain how Sean managed to achieve this (maybe he will share his secrets one day); but funds came from institutions around the world. Some were directly used to pay local TAs; this was really helpful in China for example and avoided huge paperwork for us. We also received in-kind contributions in the form of administrative support time from another summer school that had been canceled but that already had staff on payroll. Needless to say that we were extremely grateful for all this generosity that made NMA financially 100% accessible — at least in terms of access fees.

Unfortunately there are many barriers that we currently do not know how to overcome. For example, we had some applicants last year indicate that in their country they had frequent power outages. Some indicated that they only had power for a few hours every day. Of course that also meant no internet access. And slow internet connections was another frequent worry. I must admit that while we encouraged those applicants to do what they could and try to participate, we had no means or ideas regarding how to effectively overcome these barriers. Maybe one day we can send those participants a limited duration satellite internet dongle? And provide them with funds to purchase a power generator?

An implicit but important barrier is access to the information that NMA even exists. Our professional networks turned out to be very North America / Europe centered. And since most of our communication with the outside world was through Twitter, we were unlikely to reach the rest of the world if they were not already aware of us as researchers. So we created an outreach team with the goal to spread the word. And spread the word they did. They managed to contact at least one institution in every single country in the world to let them know about us. However, time did not permit us to do this properly: by this time it was early June and the launch of the school was only weeks away. We had trouble identifying existing relevant networks in Africa and Asia to take advantage of already established programs to spread the word about NMA. There is definitely still much opportunity for improving further outreach and diversity.

Some of the other barriers that currently exist might be reducible in the future. For example, if we could help with the loss of income or to hire a caregiver/babysitter by providing a form of scholarship to participants in need. But unfortunately we’re not there yet. We still have many avenues for improvement to work on and fundraise for.

One avenue that we did successfully pursue was overcoming geo-political access restrictions. For example, we made sure that our teaching materials were accessible to participants in China. In China access to Google (we use Google Collaboratory), Youtube (lecture videos) and Github (tutorial content) is blocked and accessing them through unauthorized VPN connections can get you thrown in jail. So we had a whole China tech stack team that worked out alternative access solutions for our Chinese students. This was a lot of extra work; accessibility is not something that comes easy or cheap, but it’s totally worth it!

To illustrate how gratifying this can be, let me tell you about our Iran situation. So we were going to have an online summer school accessible to anyone, including Iranians. And we were very happy that this would be possible since the Iranian neuroscience community is quite large and always struggling so much trying to be part of the international research community. This struggle is of course a direct consequence of the US (and other countries) sanctions against Iran. But since we were online, access for them would be easy — providing they use VPNs to circumvent Iran’s IP blockages, but apparently that’s not a big deal (at least that’s what our Iranian friends told us). We had about 80 Iraninan students and 8 Iranian TAs all excited and ready to participate. A local business generously offered space with internet access for the TAs. We were all set. Or so we thought…

A week before NMA started, our lawyers informed us that we would not be allowed to teach to Iranian students or have Iranian TAs, not to speak of actually paying them. This would be in direct violation of the US sanctions. Providing services or goods (money) to Iran residents could put all NMA organizers and lecturers at risk of legal pursuit and jail. This meant that we had to cancel all Iranian participants a week before NMA started. Imagine the disappointment and rage against us. How could we? Shouldn’t we have seen this coming? But at that point, we really had no choice. This was probably the darkest day of NMA. Twitter went crazy over us announcing our own disappointment. Lots of people were furious with us that we didn’t just do it anyways. Some suggested we should cancel the whole event in protest. The support and empathy for our Iranian community was heartwarming. But still, we felt like we had failed the community.

Let me tell you a bit about all the things that went through our minds during those days. (And now that there is some water under the bridge, I think it’s ok to share some of this.) We were in an instant board of directors emergency meeting trying to figure out what to do. This was not one of the fun ones. We were disappointed, sad, upset, furious — all at the same time. We pondered different ideas that were desperate attempts of finding a solution to a problem that could not be solved by us. Some obviously ridiculous ideas involved asking Iranians to lie about their location and re-register, which of course would have put us all in violation with the sanctions. We fantasized about secretly traveling to Iran with a suitcase full of dollar bills to pay Iranian TAs. And this fantasy was quite elaborate, if I may say, involving trench coats, hats and dark sunglasses… you get the picture. But as I said, these were just daydreams out of desperation and we never seriously considered them of course.

There was, however, a real though unlikely potential solution to our dilemma. At the same time as informing us we can’t teach to Iranian residents, our lawyers also informed us that we could apply for a license from the Office for Foreign Assets Control (OFAC) of the US Department of the Treasury that would allow us to potentially teach to Iraninans. There was an “emergency” fast-track procedure, but chances for this to happen within a week were so low they might as well be nonexistent. We wanted to try anyway. So our lawyer team prepared the paperwork for us and we submitted the application a few days later with little hope of success. In the meantime, the damper that this news had put on NMA was palpable. Within NMA there were many critical voices asking why we did not know about this earlier. Some of our organizers left NMA out of disappointment. Outside of NMA we were accused of treachery by very prominent colleagues (I will not name names). This was affecting us personally and NMA as an organization in the most profound way. We had failed at what we had worked so hard at trying to achieve: being maximally inclusive.

The cherry on top of the cake was that we had already internally done the group assignments and TA assignments including all Iranians and — as we wanted to promote cross-cultural interactions and networking — these groups were all mixed with non-Iranians, at least to the degree that participants did not select preference in instruction in their mother tongue, Farci, which we offered. This meant that our team had to redo all the group building and matching to actively remove Iranians from NMA. Talk about a heartbreaking task. That week was no fun. We were already stressed out and stretched beyond the limit this being the last week of NMA preparations. We did not need this extra work on top of all the last-minute unanticipated and/or piled up tasks that needed to be completed.

Then a miracle happened. Sunday night Eastern time, the day the NMA had already started in the Asia/Australia time zone (because they were the first time zone globally to start the Monday morning NMA kick-off) we received an urgent email from our lawyers. Coming in at around 10pm, they informed us that we had just successfully obtained an OFAC licence to teach in Iran. I was already in bed sleeping; but luckily within NMA there are always people awake and ready. So a whole team of people pretty much worked all night long to re-integrate our Iranian students and TAs into NMA. I leave it up to the reader to imagine the immense joy for us NMA organizers, not to mention the overjoyed reaction of our Iraninan friends and NMA TAs and participants. This was a glorious day for science.

Obtaining an OFAC license (see Nature news coverage here) has resulted in a lot of positive press and reactions on Twitter. But the most important aspect for us — aside from being able to include our Iranian friends and colleagues — was that overcoming sanctions was absolutely possible if you only tried. For as long as I can think and for every conference, seminar or summer school, there was always the problem of Iranian’s (and any sanctioned country for that matter) participation. And organizers would always just shake their heads and say sorry, we’re not allowed to have you participate. Clearly, there is a way but we realize that we got lucky and this process can take years under normal circumstances! This was a big lesson of success for us. We should never take “no” as an answer. Where there is a will, there is a way. Now the question just remains whether there is a will…

We will not rest on this success. There are other sanctioned countries on the OFAC list and we have already applied for an expanded license to teach in those as well — minus North Korea maybe, because it would just be technologically and logistically impossible to reach people there, unfortunately. And as I said there are other ways we are thinking about increasing accessibility, inclusion and diversity. For example, we would like to reach more people in underprivileged regions around the globe. This will be a big priority for NMA 2021. But one thing I have learned from CoSMo is to never rest on your successes; there is always room for improvement. And not evolving and improving means regressing. So we will continue to push the boundaries through outreach and means to improve accessibility of NMA.

5. Building and testing the curriculum

So we were as inclusive as we could given the tight 3-month timeline. Now we just needed a curriculum. Our starting point was of course the new CoSMo plan that we had worked out during summer 2019. We wanted to give basic instructions of traditional computational neuroscience approaches and also include modern data neuroscience (i.e. machine learning) techniques. Most importantly we wanted to teach the meta-science level: how do we ask good questions, how do we build meaningful models, and how do we ensure interpretability? We wanted to make sure that we teach not just how certain computational tools work (you can find that in many textbooks and online tutorials), but also — and maybe more importantly — how “hot topic” analytic tools relate to reality, how theories relate to data, or relationships among disciplines. We thus needed to find a way to teach the policies, procedures, and processes underlying computational neuroscience research.

This was a very ambitious goal, something that had never been attempted in an online format. In fact very few in-person summer schools even consider this explicitly. But we had great success teaching this at CoSMo and so we wanted to ensure we can transpose this into the online world of teaching. The way we achieved this (I think) was multi-pronged. First, we wanted explicit instructional modules focusing on the meta-science of modeling. Since a deeper understanding of what models are and what they can do for us would motivate the whole school, we placed these at the beginning of the summer school. As a result, the first day introduced different kinds of models and how they can answer different questions. But of course in real life it works the other way around, i.e. you first have a question and then need to design a model to help answer that question. So we needed to teach how we can get from a scientific question to a model that answers this question. This introduction to how-to-model became our second day of instruction. And that latter day was not perfect. By far. We had written a paper about how to model. And we would teach this in person at CoSMo every year with tight mentorship. But turning this into an online self-study tutorial was very challenging.

The second way we tried to teach the meta-science aspect is to have participants work on small group projects. In that way they could apply the how to model instructions directly to a real research question. This is how we did it at CoSMo, but there, we instructors would provide very detailed and direct in person feedback and mentoring on the process individually to each group. This was of course not possible for NMA group projects since we had not just a dozen but several hundred projects (despite them being an optional part of the NMA curriculum). How did we approach this? We asked TAs to provide some limited help with projects. But that was not very successful, mainly because we failed to provide clear instructions to TAs on how to guide the student groups. We also had external mentors (experienced researchers) donate their time to provide project groups with some limited advice and guidance. This worked better but was still very variable in quality. Again, we think that this was our fault since we did not provide mentors with specific guidance. There is definitely lots of room for improvement here. However, overall the projects were a success. We had some great curated datasets that students were able to use for their projects, we had some specific guidance videos, e.g. on how to brainstorm. And most importantly, participants had fun with the group projects and built some lasting memories. This was clearly apparent in the project presentations at the end of NMA; many projects were of high quality and students had learned a great deal.

The third way we taught meta-science was through the actual instruction materials. We were careful to emphasize for each topic not just how it works, but what kind of questions could be answered with a given approach or toolkit. This was mainly communicated in the introductory lectures and also in the more general context providing wrap-up lectures at the end of each day. As a result, our instructional focus was not on the methodological details but rather on the scientifically relevant aspects of a method that would determine interpretation, explanation, and justification of findings. Our philosophy here was that if someone wanted to learn all about how a technique worked, then there were lots of textbooks out there to read up on it. However, there are no textbooks explaining when and how to use a given toolkit to answer scientific questions. And how do you interpret the results correctly, what are limitations, etc. This is what we attempted to convey. And I think our lectures and tutorials generally achieved this goal.

Now aside from the meta-science aspect, how did we actually decide what to teach? After all, the field of computational neuroscience is vast and there are many basic and advanced techniques out there. As I said earlier, we essentially integrated 3 approaches. Our starting point was the redesigned CoSMo syllabus. But we also wanted to make sure that we catered to the community and met attendees’ expectations. Therefore we asked Twitter for suggestions on topics and what people would like to see covered. This gave us a long list of topics with a surprisingly high overlap with our CoSMo plans. But there were also a series of topics that we had not planned to cover but that seemed important to many in the field. We thus collected and organized all suggestions and created a curriculum committee within NMA that used all this information to propose the best possible curriculum.

Curriculum design is by no means an easy feat. Especially when it’s done collectively in a multi-disciplinary crew. Everyone has their own perception on what’s the most important topic to cover, which ones need in-depth instruction, which ones to leave out because they’re too advanced or too esoteric, etc. Also, what’s the best order to teach materials in? Should it be ordered by math concepts or difficulty level or topics or something else? Let me just tell you that we had many heated discussions over this… Whenever I thought we had made 2 steps forward toward converging on a final curriculum, someone dismantled it again and we stepped 3 steps backwards — or this is how it felt, especially given the insane time pressure. Democratic processes are hard! But within a couple weeks we managed to come up with a tightly packed schedule featuring 15 day-topics of instruction with six big tutorials every day covering the foundations of each topic.

Of course Konrad immediately realized that we had no idea if this was feasible. So we decided early May 2020 that we needed to run a test day to check if the general structure we had planned was feasible. And so on May 22, 2020, we had “Bayes day”. We chose to test run the Bayesian statistics day, mainly because Konrad was leading the development of that day and volunteered to roll it out fast. We recruited (through Twitter again) a group of volunteer students from our TA applicants; they would work through the day and give us feedback on what worked and what didn’t. At that point, a day would start with a big lecture followed by the tutorials. Each tutorial had a tutorial intro lecture and then a series of exercises. Finally there was an outro lecture wrapping up and providing some outlooks.

Bayes day feedback was decimating and super instructive. The most important lesson was that we had packed way too much material into a single day. We needed to cut the planned NMA content by a third! The other super important piece of feedback was that the current structure of having a single tutorial intro lecture followed by a series of exercises did not work. We needed to break things down into much more bite-sized elements so that participants would not get lost. So we had to essentially redo the whole curriculum and adapt our plan on how to deliver the tutorials. We now had to create a mini instruction video for each tutorial step. Tutorials needed to be limited to 1-line code additions by the participants in order to keep them feasible within the allotted time. And we needed to streamline tutorials so that they’re self-contained and do not require expert guidance (this is because our TAs would not be experts in all topics). There was a lot of work to be done!

At the same time, we scrambled to find good lecturers and tutorial developers. This was a big commitment and will become clearer below. First, we recruited a “day chief” for each instruction day. Day chiefs had to oversee and coordinate the content generation for each day. They could rely on “one-pagers”, a simple outline developed for each day including prerequisites (to ensure they would be covered by previous NMA days and/or course prerequisites) and how they would be attained, learning goals, lecture materials to be covered and tutorial content and goals to be developed. Day chiefs then had to theoretically find and recruit lecturers for intro and outro lectures and tutorial developers and accompanying instruction video lecturers. While some day chiefs did great, others required a lot of “help” from Konrad and me. Many personal “reminder” messages were sent, bribing was involved, and asses were kicked (the nice way of course). It was really hard to have people understand and respect our tight timeline (more on the timeline below). But in the end it all worked out.

And as I mentioned earlier, through Bayes day we also realized that we needed to test and curate our teaching materials extensively to be appropriate for a very large online audience. This was the time when Patrick deployed his project planning skills and made a detailed timesheet outlining all the things we needed to do, in which order they needed to be done, and when they needed to be done. This was — as I already said — one of the most crucial exercises we went through. We realized that creating the lecture slides and tutorials was only a small part of the work that needed to be done. More about this later. Most importantly this planning exercise gave us a very tight timeline for producing lecture materials. And that timeline was determined by the “prepods”.

We called our groups of students “pods”; each pod consisted of ~10 students paired with a TA. And since we wanted to test run all teaching materials before NMA, we called this our “prepods”. The goal was to roll out all materials to 2 groups of 10 or so students and two teaching assistants (TAs) — one for each group. At the end of the day, each group would discuss how the day went and come up with a list of recommendations for improvement. Each group would then meet with the instructors team to provide direct feedback and allow for a discussion to happen. We ran prepods in June 2020. We had 2 pods, one doing weeks 1 and 2 and the other doing weeks 1 and 3. This is because weeks 2 and 3 materials are independent from one another but both require week 1. It allowed us to speed up the testing by one week. A single week was a huge time gain during this insane crunch time.

Running prepods meant that all lecture materials needed to be ready and as polished as possible to get the most useful feedback. We essentially had 3–4 weeks to produce all lecture materials, record and video-edit the lectures, design and implement the tutorials, make and record the tutorial instruction / explanation videos and upload everything to Youtube and Github for the prepods to access. And all this was knowing that we had to redo it all after receiving prepod feedback. Good times. Most of us were not used to this kind of iterative approach. Usually we make our lecture slides the night before the lecture or seminar and hope for the best. For in person events, being present and reacting live to the audience and questions allows us to cover up mistakes, correct logic flow, and adjust to the audience’s interests in real time. But this was not possible here since all materials were pre-recorded and asynchronous. Therefore testing and improving materials was an important exercise.

Prepod feedback turned out to be crucial. Many bugs and lack of clarity were identified, some materials were way too long and needed cutting and/or shortening. There were logical mistakes, places leading to confusion, concepts that were not introduced, and many more issues that were highlighted. Each day’s team came out of the debriefing with a long list of modifications to be made. And then the work started all over again for the content teams. But importantly, this is also where the biggest behind the scenes work took off.

6. Polishing the teaching materials

When we realized that we would need to test and polish teaching materials, we did not imagine the huge amount of work this would present. After all, preparing for an in-person class — while taking a lot of time — is feasible and we all have a lot of experience with that. What we did not have any experience with is to produce a good quality video recording. Just look into one of your latest seminar or lecture recordings that did not get edited. How many times did you say “umm” or correct yourself? How many awkward silences were there because you lost track of your train of thought? For those of us who record lectures, how many attempts does it typically take you before you’re half-way happy with the result? For in-person lectures, if something is unclear students just ask and you can explain. Recorded lectures do not offer this opportunity and thus need to be watertight. Achieving this represents a significant amount of additional work. In the end, we estimated that a good half hour long recorded lecture took about 40 hours of preparation, scripting and recording. And that was just the beginning of our process.

Once the first version of the lecture materials were designed and recorded, prepods gave extensive feedback for improvement. This included feedback on the design, content and logic of the slides as well as feedback on the lecturing and logic of explanation. After that, we had a copyrights team inspect each and every slide deck to identify potentially copyrighted materials. Remember, our materials are available under the CC-BY license and this means that all materials also needed to be in the public domain. This prevented using any materials directly copied out of for profit journals for example. Not only did our copyright team highlight figures that were of concern, but they also — amazingly — offered to help find alternatives. In some limited cases where no suitable alternative could be found for crucial illustrations, they even redrew whole figures. This was a huge amount of work and we had dozens of volunteers help with this task.

We also realized that sound quality was crucial to understanding the video content, especially for non-native English speakers. We thus extensively tested a whole array of commercially available microphones and suggested to lecturers to purchase one of the “NMA approved” units. We would of course reimburse them for that if they wanted — most lecturers however did not submit a claim.

Our initial recordings were done by simply using Zoom. This worked well but did not give lecturers enough control over placing the lecturer’s video at the right size in the correct spot (upper right reserved corner) of the slides. We thus suggested the use of Open Broadcasting Studio (OBS). OBS is a fantastic open software; unfortunately it can be quite challenging to install and configure on a Mac and the mental barrier of using such an elaborate program was large. While some of us effectively used OBS thus rendering video postprocessing much easier, most continued with Zoom recordings.

Once the final, corrected and improved version of the lecture slides were ready and the updated lecture was recorded, another huge processing pipeline started: video editing, sound optimization, closed captioning (in English, Mandarin and Spanish), title slide additions, uploading on Youtube (including adding video descriptions and credentials), and linking to the videos from the Github course page and tutorials. Just looking at this long list of sequential post-production processes gives you an idea of the amount of work this would be. The video editing team led by the amazing Tara van Viegen was our biggest team with over 100 volunteers working day and night on perfecting our lecture videos.

One can only imagine the tight orchestration of preprocessing steps. As soon as a recording was ready, lecturers would upload it onto our Google Drive and indicate in a spreadsheet that it was ready to be polished. Then the video crew moved in and did their magic; they removed all the “umms” and pauses, cut out the mistakes and kept the retakes. They also enhanced the lecturer’s video as needed and scaled and positioned it correctly when necessary, since Zoom did not allow for precise control (see above). They enhanced the audio quality by normalizing volume, filtering ambient noise and correcting for distorsions. Once the video was in good shape, it was uploaded to our NMA Youtube channel. Youtube automatically added closed captions (CCs), but they needed to be manually corrected in many places since technical language was often misinterpreted. In addition, our translation team took all CCs and translated them into Mandarin and Spanish, the two most prevalent languages among our participants aside from English. CCs were used by over a third of our participants and feedback indicated that they tremendously helped with understanding.

The final step of video editing was to add a nice splash screen to the beginning of every video showing the lecturer and video title along with an indication of where this video belonged in NMA, e.g. this is the week 1 day 2 intro lecture, etc. Once all the above steps were completed, the video we re-uploaded onto Youtube and detailed credentials were added to recognize the tremendously hard work of our volunteers postprocessing each video. We put a lot of emphasis on acknowledgments. Not just on Youtube, but also in all other materials; it was really important to us that we properly credit all our volunteers behind the scenes for their hard work.

The other part of polishing teaching materials consisted in perfecting, harmonizing and streamlining our tutorials so that they were as coherent as possible across all of NMA. We called this the tutorial waxing process and it was absolutely critical to our mission of providing high-quality tutorials that are standalone and doable. So what was waxing all about?

Our waxers were a relatively small group of very experienced programmers. They were led by the amazing Michael Waskom (who developed the Python package Seaborn) and Ella Batty. Their team took the messy tutorials that our content creator team designed and made them usable. They developed standards for the review pipeline, standards for the content and ways to hide unnecessary details in helper functions that were accessible to the interested person but hidden away to streamline the tutorials. Their meticulous work ensured that tutorials are appropriate in length (e.g. word count limits), all look and feel the same (e.g. same structure and formatting), all are organized in a logical fashion, and all provide a “solutions” link to a separate page so that no one could get stuck. Waxers also ensured instructional micro-lecture videos were properly embedded in the tutorial notebooks. And they turned part of the coding exercises into interactive demos making use of widgets to enhance the learning outcome.

Ultimately, waxers did something much more important though than polish the aesthetics of our tutorials; they ensured that the learning goals for each tutorial were met. We did not want participants to get stuck on coding issues; rather, we wanted them to engage with the scientific content of the lessons to deepen their understanding. The coding portion of the tutorials thus had to be simple from a programming perspective, but at the right level of challenging from the scientific point of view. We wanted participants to reflect on the approach, think deeply about why and how something should be done and what the method and/or result implied. To achieve that, waxers helped to decide what hands-on coding participants would benefit from and what parts would be better as an interactive exploration tool using widgets. They also helped streamline the code that participants were exposed to by hiding away coding details that were unnecessary for the understanding of the exercises so that participants could focus on the important bits of code. Hidden code could of course still be inspected by interested (advanced) participants. But by streamlining the amount of code, waxers made sure that participants could focus on the learning objectives.

Waxers also organized the content review for each tutorial. Every tutorial was inspected by experienced modelers for coherence and completeness of the content and extensive feedback was given to the tutorial creators in a special Slack channel for each instruction day. Content reviews ensured learning objectives were met, prerequisites were complete, length and difficulty levels were appropriate, and suggesting which parts of the tutorial should be made into “bonus” content. After this tutorial review, the above-mentioned content editing happened to ensure standardization of tutorials, clarity and accuracy. Once everything was ready, waxers would merge the content into our Github’s master branch. This included a detailed automated continuous integration pipeline for a final extensive quality control step to ensure the whole notebook ran smoothly and contained all the required elements. In summary, waxers were the reason our tutorials and effectively our whole summer school were a success.

7. The devil is in the details…

Sounds like a marathon? It was! These were very hectic times. Remember, we only had 3 months from the original idea to the start of NMA with everything to build! From the team to the plan to the materials, including test runs and improvements. And while this was not smooth sailing by far, some of the biggest headaches we encountered were in the details.

We used the Google document suite for everything because they’re easy and convenient collaboration tools. And for the most part that worked well. But we also had some major issues with them. For example when accidentally setting the sharing options to “everyone with link can edit” after sending out a document for public feedback on Twitter. This does automatically now give everyone edit access retrospectively. And while this is less of a problem for a Google Doc thanks to version tracking, this is a huge issue for Google Forms for example. Forms do not have automatic version control and so we had to spend countless hours to undo this mistake after people purposefully or inadvertently edited our forms.

Working with forms was even more so an issue when handling participant and TA applications. For example accidental form edits resulted in lost data. And Google has no ways to recover any of it. This was not only frustrating on our end, as you can imagine, but also very embarrassing and unfair for the applicants. At this point, all we could do is apologize and ask applicants to double-check their submissions. From this, we learned that we needed a better tool. Ideally a real online database tool that allowed for forms, was secure and did not suffer from all the Google Forms glitches. We quickly settled on Airtable and we use this now for everything sensitive and interactive.

Using the Google tools had another drawback: it isn’t accessible worldwide because certain countries (e.g. China) blocks access to Google. As a result, we had to roll out a whole different tech stack for Chinese applicants and participants, including for the application submission system. So in addition to moving to Airtable, we decided that we needed an actual Neuromatch Academy portal that would not be blocked by anyone. Unfortunately this costs a lot of money, but we figured that it was worth the investment. Also because once you have a portal account, you’re in the system and we can send you email updates etc. We are currently developing this solution for NMA2021 and beyond.

Talking about email, here is another “little detail” we had to deal with. Did you ever try to send an email to a few thousand people? Virtually all free email servers prevent you from doing this in the first place. E.g. Google has a 500 emails per day limit. So we needed another way to do this and we resorted to paying for Mailchimp. But another issue is that when you send mass emails, spam filters pick up on that and classify your emails as spam. As a result we had a lot of trouble communicating with applicants. We have now actually switched from Mailchimp to Sendgrid.

Communication in general was a contentious issue. Communication takes time and we had little time available in our frenzie, so this is — unfortunately — where corners got cut. Our web site was more out of date than up to date. This is of course not good; people check for info on the web site and if they find contradictory or no information there you get inundated with emails and Twitter messages rightfully asking for clarification. Ideally, information should first be made available on the web site before emailing or tweeting about it. Because surely people will go and check and get very confused if there is a lack of information or information mismatch.

Similarly, internal communication was also difficult. How could anyone keep track of the daily decisions and fast changing information within NMA? We had meeting notes all over the place, lots of Slack messages that were quickly buried in the other thousands of messages sent that day, and Google docs with instructions and current working procedures that were often hard to be found… dozens of daily messages on Slack just asked for where to find information. Just digging those up and posting them over and over again was a big time sink. I think ideally we’ll need a wiki to keep track of procedures. But that’s yet another thing to do and we’re already spread thin and overcommitted. Not sure what the solution is here aside from organizing our Google docs better. So we decided to continue using Google docs but to move to GSuite, since we could get that for free as a nonprofit. That would at least allow us to have decent access control.

Another somewhat unexpected devil was in the details of matchmaking. Now, the name “Neuromatch” — as you might know — stands for the fact that we use algorithms to match people with similar interests etc. This has been developed by Titipat Achakulvisut as a means to bring people together in science to create community and improve the efficiency of the collaborative endeavor of science. We planned to use matchmaking throughout NMA. We wanted students to be matched into pods based on similar research interests, TAs to be matched to groups in the same way, and mentors to be matched to projects based on expertise. What we underestimated was the very important constraint of time zones and language choices. If a student indicated a preference to be in a certain time zone (and we had 9 choices among 3 main geographical time zones), this dramatically reduced our flexibility to match students with similar interests. Moreover, we allowed for students to indicate their preferred language of instruction. We ended up having pods in 14 different languages with language-matched TAs so that students could learn in their preferred (non-English) language. However, this further constrained our flexibility to match students based on research interests. In addition, the existing algorithms for matchmaking turned out to be inappropriate for these hard constraints. As a consequence we had to rewrite some of the algorithms and in the end there was a lot of manual tweaking involved in making it sort of work. There is definitely room for improvements here.

8. Designing and running projects

Projects are an integral part of any typical summer school. And there are good reasons for that. They allow students to get to know each other better and thus act as social glue. They are an opportunity for students to immediately apply some of what they learned leading to better retention of the materials. And they are simply fun and motivating for students, especially if they receive guidance and feedback from instructors. Therefore, projects are also a great way of scientific networking. I know of many cases where project work at a summer school has led to longstanding collaborations, publication of the project work, and lasting friendship. And projects allow students to directly apply the materials taught at the summer school to real science.They’re thus a great way to consolidate learning.

One specific part of project work that we have developed at CoSMo and are now applying to NMA is the meta-science approach of how-to-model (Blohm et al., 2019). In fact, modeling sounds easy when you read a paper. Many people are under the impression that modeling is very linear and easy compared to experimental work. After all, you just need a computer and all it takes is to write some quick code to crank out results, right? That is of course a very naive, uninformed view of computational work that is fundamentally wrong. Good modeling work takes a lot of technical skill and — probably more importantly — insight into the modeling procedure and process.

The modeling procedures and process is what is rarely taught. This is what I refer to as meta-modeling. How do you get from a scientific question to a model? Where do you start? What is a good question? How do you know what is a good modeling approach? How do you know when you’re done modeling? So many questions for which you’d be hard pressed to find good answers in the literature. In fact, most modelers have learned how to model by symbiosis; the magic of the dark art of modeling has been transmitted implicitly by experienced mentors. Most experienced modelers are unable to articulate the process. What’s involved, what the different steps are and how to proceed. I was personally lucky to have gotten very explicit instructions from one of my mentors, Lance Optican, who is now enjoying retirement. Lance had a very clear view of the process and procedures and explicitly taught me to think about different steps involved. This was really the basis of the how-to-model procedure that we now teach.

So at NMA we wanted to translate this how-to-model approach — that we teach and closely mentor at CoSMo in person — to an online setting. This has been very challenging. But importantly we wanted to ensure to provide students with the right meta-modeling tools to do the projects. This is why we teach the meta-modeling aspects during the first two days of NMA. First you have to realize that there are different types of models with different goals. Then you can think about your own questions, goals and hypotheses to direct your modeling effort. This is what students are intended to do during projects. So for us, on top of the traditional goals of modeling, projects are mainly a way to get familiarized and get hands-on experience with the process and procedures of modeling. This is something that is much more important than experience with a specific set of tools because the general approach translates to any domain in science and beyond. We thus use the projects to teach a life skill to students that no other summer school teaches explicitly. This is one of the important aspects that make NMA so unique.

Of course this was our theory and goal in including projects at NMA. It did not turn out quite as perfectly as intended in the first iteration of NMA, but that’s ok. We have obtained lots of feedback from students, TAs and mentors (as well as from our own observations) that will help us improve projects in the future. There have also been very specific hurdles and setbacks that we encountered that have led to setbacks and challenges. But again, like with everything new, we will likely need a few iterations to really get projects right. And this is maybe why projects were not as optimized as the rest of the course materials yet: we did not prepod projects due to the lack of time. Thus projects at NMA were the first test of our plan.

Jumping right into projects without having had a chance to test them was of course due to the time crunch we experienced. Remember: we only had 3 months from the creation of NMA to actually running the course. The highest priority has been the lecture and tutorial materials. Projects were to be tackled once prepods (our internal “dress rehearsal” to test NMA materials and get early feedback; more about prepods below) started and we had more time to commit to projects. Unfortunately our initial project lead, Paul Schrater, experienced a tragic and terrible loss at that time. We were unsure what to do. We did not want to ask Paul about his commitments to NMA during this time of intense sadness and grief. We also did not want to just carry on without Paul’s consent; after all this was Paul’s baby and he was the ideal person to develop the project aspects of NMA given his experience developing project guidance at CoSMo. And obviously we did not want to hurt him even more by just carrying on with another project leader without asking Paul about his opinion. And we wanted of course to support Paul in every way possible. Eventually, Paul’s long-term collaborator and good friend Xaq Pitkow volunteered to take over projects with Paul’s input if he had the strength to contribute.

By now we were merely a few weeks away from the start of NMA with no concrete plan. We had discussed different project formats, from mini-projects to one big project with all kinds of combinations of the above. We had also played with the idea of dropping projects altogether. In the end, we felt projects were too important to remove from the curriculum. Teaching the meta-modeling aspect at NMA needed to be front and center; no one else in the world formally teaches that. So we decided that one big 3-week long small-group project was the way to go.

What needed to be done? Everything. The project team was in crunch time. They created project templates, and guidance videos for brainstorming. They had already selected 5 open datasets to be made available to students for the project work. Those datasets were selected because of their richness and diversity in data types. Most also included behavior, which was an important feature we wanted datasets to have — because any good model should explain behavior. Amazingly, we had a team of people (often including the original authors of the datasets) who created special loading functions for us so that the data would be more easily accessible for our students. Together with the guidance videos on what the datasets contained and how to use them, these datasets were the basis for virtually all projects.

The fact that projects were all using data science approaches was a limiting, but necessary, factor for this first iteration of NMA. Since we were in such a crunch, we thought that specific data-driven questions were easier to approach by students than more theoretical modeling endeavors. In the following years, we would expand projects to include computational modeling and theory. Improving on projects by expanding their scope is thus another way to better integrate our meta-science teachings throughout NMA.

Now that we had a plan for projects and some example project templates that students could use as a starting point if they did not have any other ideas, the biggest question was how to guide students through the modeling steps. This is why we spent the first two days of NMA on the meta-modeling aspects: to understand how to model, students would need to understand what models are all about. What are different model types, how can they answer different questions and allow you to reach different modeling goals? And once you have a question, how do you get from that question to a model? The how-to-model guide that we taught on the second day of NMA was thus central to making projects work..

The next challenge was to provide appropriate mentorship for the projects. Mentorship in model development is in particular critical at the initial stages of modeling, i.e. getting from the question and phenomenon that we want to understand to a plan regarding how to approach the modeling exercise. This is the hardest step that is rarely taught. This is something that we (organizers) would do in person at CoSMo during long hours at night. But at NMA we had almost 300 project groups that needed help! And so we needed mentors. Lots of mentors. So we asked for volunteers in the community to donate their time to help students with their projects. We were lucky to get many people who signed up for that, but we were still short in person hours. We also wanted mentors to be matched to student interests. So we took a 2-step approach. One initial set of mentors helped students in brainstorming and defining their projects in week 1. Then we re-matched mentors to project groups for the remaining weeks based on overlapping research interests. While this often worked, we did not have enough volunteer mentors to actually have perfect matches. But I think -it worked — mentors were still able to provide guidance in projects thanks to their experience and expertise in the field broadly speaking.

In addition to mentors for projects, our TAs also provided some help to students. However, we failed to provide specific instructions to TAs and mentors on how to best guide students through the projects. Again, this was because we literally ran out of time and had to cut corners. This is another way we envision changing how the projects work — providing specific guidance videos to mentors and TAs will be an important future addition.

Despite all the difficulties we encountered in getting the projects up and running, they were overall a big success. It was astounding to see what our students were able to achieve in the short 3 weeks. At the end of NMA we had students present their projects in their super-pods (a grouping of pods with a senior lead TA). This was not just a lot of fun but also really rewarding for everyone involved. We then highlighted some of those projects in our closing sessions to showcase and celebrate our students’ achievements. What was also amazing to see is that many mentors went far beyond their assigned mentoring hours and got deeply engaged with the student projects and mentorship in general. This was the kind of symbiosis we were hoping for and it was great to see that it happened, although of course only in a (small) subset of cases where the right chemistry emerged. But that was of course expected.

In the end, we were very happy with the outcomes. I think most project groups achieved their learning goals. They successfully applied the how to model step to their questions and managed to produce a preliminary answer. Many groups went way beyond that and produced original research results, some of which were later submitted to the Neuromatch Conference 3.0 that took place in October 2020 in lieu of the annual meeting of the Society for Neuroscience that had been canceled (and the virtual conference, which was postponed to January). So all indicators of project success were there. That’s certainly something about which we were very happy.

9. Training and guiding TAs

A week or two before we started prepods, someone asked: “how will TAs know what to do?” Yikes, we had forgotten about this minor detail; we needed to train TAs in being tutors. Luckily it wasn’t too late and Kate Bonnen and I stepped up to the challenge. And a challenge it was. We were going to teach a very broad set of computational topics and could under no circumstances expect that our TAs were familiar with at best half of the course materials. So how would we ensure that they were effective TAs? What could we expect from them? What could we teach in a reasonable amount of time?

Organizing the TA cohort to ensure that TAs only taught materials they had expertise in was not an option. It would have been a logistic nightmare as we simply did not have the number of TAs available. It was hard enough finding roughly 200 competent TAs. So we decided that a good TA really only needed one skill — to be a good tutor. TAs were not supposed to be content experts; rather their role was to guide the learning experience of our students. And that even without expert knowledge.

How can one achieve that? Fortunately I had been formally trained in tutoring back when I was a PhD student. In my time as a TA, I learned all about the efficacy and fun of problem-based learning; students learned by working through exercises in a way that they discovered the materials themselves. This was achieved by guiding their thinking, not by giving answers. Amazingly, it required little to no domain knowledge. Only knowledge about the direction where to find the answer to a problem or question was needed. And the tutor’s role was to direct the thinking of the student groups rather than providing answers.

This way of approaching TAing perfectly suited us. Tutorial notebooks would have the answers readily available; it was really just about guiding the students’ thought processes on how to reach the solution. So we focussed our TA instructions on how to not answer any questions directly, but rather — wherever possible — to return the question to the students in a way that guided their thinking. We walked TAs through specific examples, we explained the philosophy, and we taught them a bit about learning theory so they had a better understanding of how we learn and how learning is a gradual, incremental process.

Aside from the formal teachings, we also put some structure in place that we hoped would help TAs when they encountered problems with their groups, questions they did not know how to answer, or if they needed any advice. First, we selected a set of more senior, experienced TAs and promoted them to Lead TAs. Lead TAs were paired with about 10 TAs and their role was to provide the above-mentioned structural support. Second, we created a TA Slack so that TAs could easily exchange information and help each other out during NMA. This created somewhat of a hive mind and was tremendously helpful. Finally, TAs organized TA Q&A sessions where domain expert TAs helped the other TAs understand the tutorial content. Not only did this help with course delivery, but it also helped in creating a TA community and a feeling of belonging. Essentially, when it came to running NMA, the TAs ran the show!

10. Running NMA

After all the intense preparation time, it was finally July 13, 2020 and NMA started. Some of us had just spent the night re-integrating our Iranian students and TAs into NMA. Others were still feverishly working to finish video editing and tutorial waxing for materials as little as three days into NMA. Regardless of our role in NMA, we were exhausted but pumped by the start of NMA. The big unknown thought was this: would NMA turn out to be a major disaster? Would students drop out left right and center? Would our tech plan fail? Would we be able to deliver all content in time? And despite our waxing and production teams working to roll out materials until the very last day of NMA, it all worked out. Shockingly… There were some hiccups, some challenges, but overall it was almost too smooth to be true.

So how did running NMA work? Since NMA was online and spread across virtually all time zones in the world, we could not possibly distribute materials on the fly and thus everything needed to be prepared in advance. Lectures and tutorials were posted online in the days before the lessons happened and TAs would review the materials prior to tutoring them. What was not ideal here was that we did not have specific day by day instructions for TAs. Or very few of them at least. We had some ideas for guided discussions for days 1–2, but no specific learning objectives or content-based instructions. TAs did get together the day before each instruction day if they had questions and their TA Slack was definitely the way they made sure they had the appropriate information. But in the future, more guidance for TAs would be helpful.

So what did a NMA day look like from the organizers’ end? Well, as NMA started the production and waxing team was super busy cranking out materials for the following weeks. But since lectures had been recorded, tutorials prepared in advance, and TAs had been trained, the only active teaching component was the live questions and answer (Q&A) sessions at the end of the day in each time zone, which were meant for students to deepen their understanding. That means we had 3 Q&A sessions every day. Those sessions consistent of a host, one of the organizers and 2–3 invited guests that were experts in the topic of the day. Students would ask questions in the Q&A box and students would vote on which questions they were most interested in. This was a fun activity, but since it was at the end of a long online day with project work still to come for many, it was not as well attended as we had hoped. Regardless, those that did attend got lots out of it. For hosts, this was definitely a fun activity with lots of good and deep discussions.

I think there were several reasons why NMA ended up being a huge success. First, we were well prepared and all materials had been tested in prepods and polished by waxers. Second, we were constantly on our guard because we expected the worst. This allowed us to roll out quick fixes as soon as something ran off the rails. Third, and maybe most importantly, during NMA we had our pod temperature team ensure that students were happy. In other words, an active management of the NMA community during the school was key to ensuring NMA ran smoothly.

Pod temperature measurement, analysis and intervention was Kate Bonnen’s baby. For that she spun up a small but highly effective team. Kate’s idea was that all students needed to fill out a daily feedback questionnaire indicating how they felt about the day. This included questions about the difficulty of the teaching materials, how they got along with their TA and peers, and whether interactions were respectful and professional. The pod temperature team’s role was to perform a daily analysis of the students’ feedback. Kate automated this to a certain extent: we were looking for negative scores that persisted for more than one day. For example, if one or several students said that group interaction in their pod was not respectful. Or if there was an indication that they did not get along with their TA. We looked at the pattern. Was this likely to be an outlier? Were teaching materials too hard that day? Did someone just have a bad day? Or was the whole group affected? As soon as a consistent pattern of negative scores was detected, we decided to intervene.

When to intervene was often a judgment call. We were trying to figure out based on the combination of scores whether there was really reason to worry or whether we thought that this might resolve itself. What could have happened in that group that day? This was sometimes almost like reading tea leaves. It definitely required a decent amount of speculation about the inner workings of the groups based only on the limited questionnaire feedback we received. We often had doubts. We actively discussed each case. Typically about 10 groups or so were discussed every day. All questionable groups that did not seem totally on fire but had some individual negative score were placed on a “watch list” and we would come back to those the following days to evaluate if things had improved or not. If the following days were positive again, we would remove the group from the watch list. This allowed us to keep track of the students’ well-being all along NMA.

If things were bad in a pod due to multiple members complaining and/or consistent complaints over consecutive days, we intervened. Interventions were time consuming but of huge help. They were also challenging because of the different time zones that organizers and students were in. We ideally did not want to interrupt lecture and tutorial times so we had to find a time slot outside of those. Then, depending on availability a person from the pod temperature team would volunteer to intervene. After the intervention, we reported back to the pod temperature team on the following day to inform the team about the exact issues encountered and measures taken to improve the situation. We would then pay special attention to that pod’s daily feedback over the next few days.

Pod interventions consisted in contacting the TA and letting them know we needed to talk to the group and/or TA. We typically asked the TA when they thought it was the best moment for that. We typically first met with the TA alone to get their input on the pod dynamic and whether they had noted any issues. This individual meeting with the TA often also allied us to provide specific instructions to the TA on how they could try to improve things. Then we would meet with the whole pod and would address the issues head on. That involved trying to figure out what the problem was (in case the TA did not know) and actively discuss ways forward for the group so they can work and learn together harmoniously and in a positive atmosphere. Sometimes these interventions just needed to reset expectations of pods. Sometimes the group’s interaction was one-sided creating tensions in the group. Sometimes there was a lack of respect. Sometimes part of the group felt that the TA did not equally divide time between pod members. Most often the real underlying issue was lack of communication. Students did not dare speak up and our intervention essentially helped less confident students to get heard. Interventions typically lasted 1/2h and at the end of an intervention there was always a clear way forward. We were careful not to blame or shame. The goal was always to resolve the issue for everyone.

As I said, I believe that the pod temperature team was instrumental to the success of NMA and the mental well-being of our students. But there were other aspects that ensured the smooth functioning of NMA. The most important one was being on our guards. We constantly expected things to go off the rails. So we were looking for them actively. We communicated extensively on Slack. We asked every organizer and lead TAs about feedback. Lead TAs had a direct line to us organizers and TAs knew that this was the way to escalate issues to the organization team.

Luckily there were not many serious issues (that I can remember). Across the board running NMA was surprisingly smooth sailing. We had only a couple code of conduct violations reported throughout the whole NMA organization and deployment time and those were dealt with as needed. We had one Colab tutorial glitch on one day that had been identified in the Asia/Australia time zone early on and needed emergency fixing. Amazingly, waxers and tutorial developers managed to patch the tutorial in time for the Europe/Africa pods. So it was all good. Even the daily transfer of our tutorials and videos to the China-specific tech stack occurred without any major glitches.

The one really unexpected failure was the breakdown of Crowdcast that we used to broadcast our live Q&As. Crowdcast had been running very stably for Neuromatch Conferences 1.0 and 2.0. So it was a no brainer for us to choose this platform as it provides very elegant and intuitive features that really promote interactivity of the audience. To our surprise and astonishment, we experienced many issues with Crowdcast right from the start. Sometimes audio would not work, sometimes we were unable to go live, etc. The lack of stability was embarrassing, annoying for the host and stressful to deal with. We quickly acquired a Zoom webinar license as a backup solution. But as issues did not get resolved despite us complaining to Crowdcast and them being aware of the issue having been caused by a recent update to their software, we decided to switch to Zoom for good by the end of the first week of NMA.

NMA also reserved a few surprises for us. One really nice one was the low attrition rate and the motivation and engagement that students showed. We had tried to do a bit of social engineering to ensure student happiness and commitment to complete the course. Small group project work was one aspect of this; once part of a project, students do not want to let down their peers by dropping out. Peer programming was another aspect; working on tutorials in small groups really helped with getting to know the other pod members and developing a feeling of belonging. What was cool and unexpected was the bonding and emergent creativity of the groups. We had given our pods automatic cool names (e.g. “fancy koalas”). Part of the icebreaker activities done at the beginning of NMA was to create a pod logo. This quickly turned into a full-blown competition among pods and we saw some amazing original artwork emerge. This was when we knew that we had succeeded in creating a nurturing, positive learning environment for our students.

At the end of the 3-week course, we were happy with the outcome. We had feverishly worked until the last days of running NMA to produce polished materials and everything had run smoothly. Groups also were quite successful in their project work with some really cool ones. So when the last day of NMA came, we wanted to celebrate our students and our own success in making it all happen. Online celebrations are somewhat awkward, but we wanted to at least provide an overview of highlights to everyone, both from the organizers perspective as from the student side. Our celebration slide deck was a fun way to round off NMA and wish everyone the best for the future.

We had done it!!!

11. Challenges and opportunities

We had done it indeed! From zero to success in 3 months at breakneck speed. So after NMA ended, everyone was exhausted. The NMA Slack went almost spookily quiet for the following week… We all deserved a break.

Taking a break is also a great opportunity to reflect on challenges we had encountered and opportunities to pursue in the future. There were certainly many places within NMA that could be improved. But there were also realizations of additional opportunities to make science a better place. So let me start with some NMA-specific challenges and places of improvements here and I will then outline in the next chapter which other opportunities we or others might want to pursue in the future.

From the beginning we wanted to be an inclusive, diverse organization and a role model in those areas. We actively reached out to and recruited organizers with diversity in mind. More importantly, we did so in recruiting lecturers. We wanted students to see the diversity of the field and as a result feel welcome in the field regardless of origin, beliefs, gender, socio-economic status or ethnicity. We had all the good intentions and in a way did succeed to a certain degree. But we certainly could have done better. The challenge was that minorities in the field do get many more requests for participation and thus have to turn down such requests more. And I’m sure there were other, more systemic factors that prevented us from doing better. This is certainly an area that we need to do better in for NMA 2021 and we are actively trying. There are different ways this can be done: (1) officially recognize the importance of diversity by having a diversity chair (NMA 2021 has a Chief Diversity Officer), (2) contact potential speakers and organizers earlier to give them more time, time that we didn’t have in 2020 but do have now, (3) more proactively think about how to increase diversity across the organization.

One hurdle to diversity is time zones. Certain aspects of an organization require live events, such as executive committee meetings etc. But it’s practically impossible to find a time slot that works for everyone from Asia/Australia to Africa/Europe to the Americas. As a result it’s very hard to be truly inclusive and diverse world wide if the positions to be filled require live interactions. At NMA we did have organizers and lecturers from around the world; this worked mostly because the positions filled by people in Asia/Australia either did not require live presence in meetings or people actually tried to stay up really late or get up really early in the morning to catch a meeting. That’s definitely not ideal; however, it’s unclear to me how this issue can be resolved otherwise.

Another challenge that I already mentioned was the fact that NMA grew organically. This meant mostly that from the chaos emerged a certain structure that was based on immediate needs and people stepping up to the challenge, rather than a planning exercise laying out future needs. We were lucky in a way to have people who volunteered to commit. And it mostly worked. But we did end up with an executive committee that was simply too big (>20 people) to be able to make quick decisions that were needed on a daily basis. It also meant that it was impossible to find meeting times that suited everyone. One big improvement that we made as a result of this realization was to formally analyze NMA structural needs for the organization and define specific CxO (“Chief X Officer”) positions to cover those needs within their departments. We now implement this for NMA 2021 and — while it’s too early to fully evaluate this — I believe it has already shown its value and benefits. A smaller executive committee makes faster decisions, each department and CxO has clearly defined roles and responsibilities, and departments have a clear agenda on what to achieve. The future will tell to what degree this was a successful change.

Talking about volunteering. Many people within and outside of NMA have asked if/when we are going to pay our volunteers. To me, that defies the definition of “volunteering”. But more importantly it conflicts with our vision and role in the scientific community. We are a nonprofit organization that — like most nonprofits — is run by volunteers. No one at NMA is paid, not the board, not the CxOs, not the many volunteers. Why? Because volunteering is a privilege that recognizes that there is inequity and that by volunteering we put our privilege to work to help the less privileged. Thus, we will continue being a volunteer organization. There will however always be the need for some thankless jobs to be fulfilled that we could not possibly ask anyone to take on as a volunteer, such as accounting, or doing paperwork. If our finances allow us, we will consider hiring help for those kinds of tasks; but otherwise we will continue working with volunteers who donate their time and energy and we’re very grateful for that! In return, we now provide a long list of potential benefits of volunteering to ensure that our volunteers can fully value their experience on their CV.

Given our organic growth, another challenge was documenting decisions and preserving knowledge. As an organization we relied on google docs, sheets, etc to document and work on shared information. And while that’s very convenient for online collaboration, it quickly became somewhat of a document mess in which the most up to date information was sometimes hard to find. We had used a shared folder on a personal account (Konrad’s) as our NMA workspace. Given that we’re a nonprofit we now got GSuite, which is really helpful for automatic permissions definitions etc and to prevent accidental access or editing of documents. However, it does not fully solve the issue of keeping track of decisions and finding the most up to date information on a topic. This is an area that might still need some improvements in the future. We had considered using Notion; however we felt that making too many changes at once would run the risk of breaking what worked.

Along a similar vein our task organization was not ideal. We started using Quire and did effectively use it for a while, but only a few people actually bought into it and so it became useless for the group in the end. This was mainly because once things became busy and our schedules were all on fire no one took the time to enter tasks in Quire or check their todo lists anymore. It did however still allow some of us to keep track of the big picture. So we decided to give Quire another shot this year. We’ll see if results are better for NMA 2021, though right now it does not look like it…

Another organizational sticking point was the board of directors (BOD) role in the leadership of NMA. Since we had grown organically, BOD members automatically had assumed executive leadership positions to get things done. So we were all part of the executive committee as well. But since the executive committee was big and sometimes unable to make urgent decisions due to executive members not attending meetings and/or endless discussions leading nowhere, BOD decided for them. This was sometimes rightfully perceived as BOD overruling the executive committee. But it was really an emergency measure that we did not like employing but that we felt was necessary on several occasions to not fall behind schedule and keep things moving forward. As I said, this was mainly because we grew organically and ended up in the organizational structure we did without much thought or planning with respect to what would be ideal. This led to one of the main lessons learned: a nonprofit of our size needed some thoughtful consideration of its organizational structure. And the BOD — while important — needed to be somewhat separate from the executive committee. Most importantly though, what transpired was that there was no clear definition of roles and responsibilities of the BOD versus the executive committee. This resulted in a clear reorganization of NMA and a transparent application and selection process of the NMA 2021 executive committee with clear role descriptions for executives and BOD. Hopefully this will allow for smoother sailing this year.

As irony had it, one of the main technical difficulties arose from our namesake technology, the matchmaking algorithms. Our goal had been from the beginning to match students and TAs with similar interests and to match projects with mentors who are experts in the field. We wanted to do this automatically of course using the matchmaking algorithm developed by Titipat Achakulvisut and Konrad. However, when actually implementing this there were two major additional constraints that the previous algorithm was not set up to deal with: (1) we wanted matching to respect the hard constraint of time zone preferences; and (2) we wanted to allow for language pods, meaning that language grouping was also a hard constraint. These two hard constraints essentially had to take precedence over the topics of interest-based matching of students and TAs. It turned out that this was not only a technical issue requiring modification of the algorithm, which in turn resulted in convergence issues and many rounds of manual adjustments or re-runs to find acceptable solutions. But it was also a conceptual problem because in the end the desired topic-based matchmaking was rather secondary. To succeed in this ideal matchmaking game, we would probably need an order of magnitude more students enrolled, which is unlikely to happen. It’s unclear if there really is a better solution given the constraints. I’m not sure however if this really was a problem in the end. Groups seemed happy and functioned well.

Finally, ensuring that students had the correct prerequisites was challenging. We provided an extensive list of prerequisites and asked students at acceptance to commit to meeting those prerequisites by the time NMA started. To help with that, the list of prerequisites included specific instructions on how to meet them with links to open access materials and courses for those who needed to brush up on topics. We also organized a pre-course to help students meet prerequisites. The problem from my perspective was that students considered the precourse as sufficient to meet prerequisites. However, the precourse was meant to be a refresher, not a full replacement for learning the materials. One cannot possibly learn programming, basic neuroscience, linear algebra, probability theory and differential equations in 3 days… Clearly we need to communicate more clearly in that respect. This is to ensure that students get the most out of NMA and do not feel lost because they lack the crucial background in math, programming or neuroscience.

12. Saving the world

Organizing NMA and having helped a bit with Neuromatch Conference (NMC) made me (and the whole team) think a lot about the state of academia, education, privilege and all the things that are wrong. And since I’m an idealist, these kinds of thoughts are immediately followed by reflections of how to improve things. How can we save the world, one innovation at the time?

I must admit that I really enjoy this kind of free thinking. It’s a bit like dreaming up your perfect little world and then trying to figure out which one of these dreams to turn into reality. Sometimes I think “yes, this one is a must-do” but then after pondering it for a while the excitement seems to fade. Then there are other ideas that grow on me and only reveal their potential impact and importance with time and deeper reflection. I guess this is my own way to figure out what might be worth pursuing and what not. So let me give you a few hints at the mostly half-baked ideas we have had within Neuromatch and I will mostly focus on the ones I have been involved in so that I don’t spoil the thunder for others.

Organizing NMA and making education in computational neuroscience available to the world in an affordable way made us wonder if this could be done more. Certainly every single discipline has materials that could be taught in the NMA way. Clearly we want to expand our course offering. This year Konrad spearheaded a deep learning course that we are currently NMAifying. But there are many other courses we could offer that would tremendously help neuroscience and potential science more broadly, e.g. biostatistics, math for computational neuroscience, actual data science (data management, visualization, code sharing etc), and so forth. Our current vision is to add one such course every year for as long as we can keep the momentum going.

But we surely cannot even try to cover all domains of science. So is there a way we can codify the NMA experience and “sell” (for free) our approach to others interested in developing similar courses in other domains? Having other people copy us is exactly what we want. To facilitate that we wrote up our experience (see TICS paper here). Hopefully this write-up will also help. We now keep better track of procedures, specifically related to tutorial creation and the learning workflow management. Hopefully we will soon be able to spread the NMA approach this way. Most likely that would result in other institutions needing help and asking us for help. At scale we would not be able to provide such help for free. But institutions could potentially hire an NMA service that helps implement NMA-style courses elsewhere. We’re actively considering this option. Such a service could also help with the matchmaking aspect of course, but that’s not for me to talk about.

Now we have joked that NMA is the size of a liberal arts college. So we’re almost running a small university. But that’s not true of course; we’re not accredited to grant degrees, we don’t have employees, and we don’t have a program (yet) either but only one course (well, two courses in 2021). Yes we will expand our course offerings, but does that mean we will become a university? In fact many people have asked us this question. And naively thinking, this is a very appealing idea. Wouldn’t it be amazing to offer an official degree to students from all around the world while being accessible, affordable and geopolitically neutral? Imagine the potential opportunities for students who could never afford to apply to universities outside of their country of residence, pay for the visa costs and move to the country in question? Not to even mention all the students who need to support their families financially, emotionally and physically. And there are all kind of other hurdles I mentioned above due to discrimination and identity that might deter students from even trying. NMA university could easily overcome that. So what speaks against it? As a university, we would not be able to rely on volunteer work. We’d need a real administration and staff just like a real university. And that is very expensive; employing just 100 people at $100k/year… well you can do the math. Compare that to our 2020 budget of roughly $400k and you get the idea. All of a sudden we’re not affordable anymore. That would totally defy the purpose. We would lose funding from all the amazing organizations that see a value in us doing this on a volunteer basis. We’d also have to quit our real jobs at our current universities because now we would be in conflict of interest. It’s unclear whether we could still draw the crowds with higher costs and the whole idea would thus collapse. I’m not saying we have explored this in all depths, and you should never say never, right? But for now, this is not a direction that we see any promise in pursuing alone; we are, however, exploring potential alternative avenues, such as offering degrees in partnership with universities around the world. Only the future can tell to what degree such a thing will become a reality.

Neuromatch wants to democratize access to knowledge and education. We teach, students do projects, we host conferences. And sometimes student projects end up in conferences or as publications. Wanting to democratize access to knowledge, it would only make sense to have our own publication venue that we could shape the way we envision scientific publication should work, i.e. controlled by researchers, not for profit, innovative etc. Of course, since this would be a “Neuromatch” journal, we would want the matchmaking algorithm to take a central part in it. We would use it to automatically find the best editor and also to suggest appropriate reviewers from a database of volunteers. Creating a journal is also a direction we are actively working on. And it’s fascinating to start learning about the publishing industry, the backend of publishing, different publishing models, etc. Hopefully Neuromatch will soon have its own journal…

Yet another way that the matchmaking algorithm can help science advance faster is by suggesting who should collaborate. People on Twitter and at conferences (and probably many other places) have talked for a while now about having a “science Tinder” that suggests who best matches someone’s research interests. Now Tinder itself has many issues that we would not want to bring into science, such as it being based on the looks of people (this would be a source of racism, sexism etc in science). But the idea would be to have a scientific community organization tool that allows people to find collaborators and maybe even strongly suggests who everyone should be collaborating with. Maybe the latter could include some incentive to actually follow up. Maybe there could be some incentives to do more collaborative / group science. Definitely, there should be suggestions of tools and procedures on how to effectively and efficiently collaborate together, especially over a distance. Regardless of the details, I believe that having such a community matchmaking and collaboration tool would be tremendously useful; after all, working together on a problem is not just more fun but also leads to more creative and faster solutions. This would be all benefits for individual researchers and science as a whole.

So why have we not done all of this yet if they’re such good ideas? Mostly because of time. Each of them might sound easy enough and quick to do, but they’re not. There are many factors to consider for each, they all require financial support to be found, and most importantly they require committed volunteers to spearhead and lead the effort. If you’re reading this and are interested in pushing some of these (or other) ideas, please don’t hesitate to get into contact with us. I believe that we can find money. But finding dedicated and committed people who will volunteer a significant chunk of their time to make our scientific community a better place is not as trivial. And I’m not judging here. We all have different constraints. For example an early career assistant professor not focussing on getting their research program going might risk losing their job getting sidetracked by something that might not be valued in the tenure process, etc etc. I don’t need to get into all the different limitations; there are too many really to list. In a way this is unfortunate; science should care more about its community. The problem is the incentive structure that we have little control over. It is what it is and we have to work with this reality. My hope is that more people will realize the value of the Neuromatch movement as we keep doing things and that some of those people will have the ability and passion to get involved. I’m very much looking forward to expanding Neuromatch further and changing the world.

13. Some personal lessons learned

Neuromatch is about love. Everyone is passionate and deeply cares about the field, about our future as a community, about social equity. When I say we want to save the world, we really do, in our very tiny little way. And with lots of love, especially from Konrad! And that love translates of course to lots of time spent working on Neuromatch activities. “It won’t be much work…” has become our continuing insider joke. But it definitely is by far the most rewarding thing I have ever done in my work life. And it’s so much fun to work with all these amazingly motivated and kind people in Neuromatch. That does not mean it’s always easy and smooth sailing though. Personally I have struggled on many ends at different times throughout the Neuromatch adventure. So I wanted to share some of those struggles and sometimes unexpected lessons I have learned over the past year.

Let me first start with a very personal one. As most of you know, I have a very blunt style of communication. I usually say things the way I perceive them and am very honest about what I say. I don’t know if that has contributed to my Viking image, but that’s a story for another time. I believe my bluntness allows everyone around me to always know where you’re at. I don’t play games. I’m upfront. I tell you when I don’t like something. Straight up. And of course I have realized in the past that this is not necessarily compatible with all cultural norms, but I had thought that people would understand. Well I was wrong. Turns out my bluntness can really hurt people’s feelings and can lead to the perception of me being condescending or dismissive. I don’t want to explain specific situations in order to maintain the privacy of people involved. But I want to say very publicly that I am truly sorry that I have hurt people this way.

It turns out being perceived as being in a power position does not help with this situation. To me, Neuromatch has a flat hierarchy in terms of personal power — because we’re all volunteers so I really have no power over anyone — but that’s not how people perceive leadership positions. In reality leadership positions in a volunteer organization are more of a service that they are power positions. They come with a lot of responsibilities. But again, this is not often appreciated by people who are not in leadership positions. So I had to learn the hard way that I have to be more careful communicating, especially because I am in a leadership position. This is something I struggle with because it’s not my natural way to communicate. I’m trying very hard though and I hope that fewer conflicts will arise because of me.

Another surprise to me was that leadership is damn hard. It’s not about directing people, it’s not about telling people what to do or how to do it. Leadership is about creating a positive and nurturing environment that is conducive to people’s growth, fuels their self esteem, and provides a rewarding experience. Transmitting the feeling of being appreciated is key. All the amazing results that NMA achieved were thanks to all of our volunteers’ — each and every single one — hard work and devotion to our vision. And in many ways, as I explained before, we got lucky that the positive energy from the leadership just kept spelling over and infecting everyone else. But sometimes that was less the case. It took me quite a bit of reading and introspection to figure out that a big part of leadership is to make sure we do create a nurturing environment, that we do give back to our own community, that we do share success stories outside of the board of directors or executive committees, and that we do acknowledge often that all these success stories are a direct result of all our volunteers hard work.

Leadership of a nonprofit volunteer organization is particularly hard. It’s not the nice salary and benefits that would motivate people because there is no such thing at NMA. No one gets paid for the volunteer work. It is thus even more crucial to create a clear and transparent organizational structure where everyone can thrive. Clarity in roles and responsibilities is also paramount; that’s not to create hierarchies, but rather to ensure that frictions do not arise. Again, being in a leadership role means having more responsibility, not having more power. This is an insight that I wish our politicians would have had more… I’m glad I got to learn and appreciate this lesson after only a few months working for NMA. I believe it is crucial in ensuring that the NMA movement continues to ignite the fire in people and spread the love widely.

A really hard aspect of leadership — especially for me — is communication. It’s so easy to take in the information from the organization’s base and then make decisions etc without properly communicating the decisions, the rationale behind the decision, etc. It’s very easy to forget during busy times that communicating these things allows everyone to buy into the mission, to feel heard, to develop a feeling of belonging. Communication is really hard. It’s easy to forget crucial details, it’s even easier to get misunderstood. This is why any organization really needs a communication officer. Communication is not a thing you just do, it’s something that you learn how to do right. It’s about the right amount of information as the right level of detail. It’s about acknowledging all the volunteers’ contribution to success stories. It’s about transparency. It’s about developing trust. I definitely still have lots to learn in that respect.

Those are the hardest lessons I have learned from NMA. They have made me question my involvement in NMA; because I felt unqualified and destructive at times. Hopefully my contributions to NMA outweighed this.

NMA has also had some extremely positive lessons for me. Most of all, it has given me an incredible amount of hope in humanity in these difficult Covid-19 times. NMA really has been for me and for many others a bright light in a time full of fear, uncertainty, pain and despair. So how did NMA (and Neuromatch in general) have such a positive impact? I don’t have the full answer to this, but I believe there are a few reasons. First, the portrayal of academia, especially on social media, is often very negative. Abusive labs, inappropriate incentive structures, exploitative environments, lack of funding, elitism, and so on often dominate the academic climate. NMA has shown that we can create a positive, nurturing and loving work environment. Second, lack of equal access to education and resources around the world leads to much inequity. NMA actively overcomes that. Third, in this fast paced world where individual success is celebrated and egocentrism in science is promoted it often seems like science has lost its soul of being a global collaborative endeavor. NMA successfully brings this ideal back. And finally, and maybe most importantly, in a world driven by personal greed and capitalistic thinking, NMA has shown that there are other ways to be successful. Ways where the people are put first, where the privileged take care of the less fortunate. Personally I was very positively surprised that this approach resonated and still resonates with so many people. What a message of hope for humanity!

They say every change is an opportunity for improvement. The pandemic forced the world to change. And I think academia needed such a forced change, although the circumstances were of course terrible. But this forced change resulted in the realization that the status quo was not set in stone, that we had been neglecting many possibilities opened up by current technology. The pandemic also forced me and many others to rethink what really matters. Is it more important to crank one more paper out or should I rather help democratize education and access to knowledge for thousands around the world? This was a really important lesson to me. It almost seemed like I was blinded and now could see more clearly what really matters. And I have since prioritized NMA and Neuromatch-related initiatives over everything else.

NMA has also taught me to dream big. Think outside the box, imagine your ideal world. How can we work towards making that happen? I have for many years said that I only want to collaborate with people that I also enjoy to hang out with socially. Because life is too short to forget to have fun and hang out with friends. NMA is exactly that. I feel like I hang out with friends while doing something good for the world. Board of director meetings feel like sitting around a campfire and dreaming up solutions to the problems of the world. Only that contrary to fireside fantasies, NMA and Neuromatch work hard to make these dreams become reality. And that really led to my biggest and most powerful lesson in this adventure: big dreams are achievable! And they’re worth pursuing.

14. Final words

In the end, after three months of relentless efforts NMA happened and miraculously, everything worked out. We had contingency plans in case TAs got sick, but no one did. We expected tech to fail left right and center, but aside from minor glitches everything ran smoothly. We feared students would drop out left right and center, but only very few did. There were no catastrophic failures despite this being a human endeavor with all the known failure modes. Somehow magically we all had each other’s backs. And this tremendous teamwork paid off.

I’m definitely proud of what we have achieved together. It was not always easy, there were conflicts, tensions, struggles and a constant race against the clock. But together we did it. Together we can do anything, it seems. I feel extremely fortunate and honored to have worked with such an amazing team. What a privilege! Thank you to all the NMA team for being so amazing! You have all turned a year of darkness into a year of light and hope for science. And NMA has helped me cope with the pandemic. I’m not sure what would have been without NMA…

In my mind, NMA 2020 was just the beginning. I have so many hopes and dreams for science. We have grandiose plans with new ideas and directions emerging weekly. There is lots of work to be done, but that work doesn’t feel like it’s work. It really is a lot of fun. And that fun is all accomplished by an amazing, idealistic volunteer community working hard to affect positive change in academia and beyond!

So my 2020 summary? Lots of hope and a new vision for academia. Who would have thought…?

Further readings and perspectives

I have mentioned many other sources of information and write-ups throughout the story. Here are a few additional articles that are fun to read and convey a different perspective of NMA.

• If you would like to hear about the NMA experience from the perspective of an interactive track student, this is a fun to read Medium article.
• Here is a write-up by Ashley Juavinett for the Simon’s Foundation who helped funding NMA.
• One of our TAs, Richard Gao, also wrote an excellent blog post about NMA.
• And colleagues in the field have written a very nice Lancet article about us.