How to Have Real-World Impact: Five “Easy” Pieces

Because my research group has had a pretty good record of getting the fruits of our research adopted “in the real world”, I often get asked how to get work adopted. I will be honest: I’m not exactly sure.

My colleagues and I could have just been lucky, but we seem to have been lucky a lot. Maybe we’ve been doing something right. Here are five pieces of actionable advice: (1) scratch an itch, (2) build real systems, (3) embed yourself, (4) give great talks, and (5) go to the mountain.

Scratch An Itch

Widely-adopted systems solve real problems. So, the first key to adoption is to identify such a problem and solve it. One way to identify a real problem is to identify a problem you yourself are facing, and then work on it (“scratch that itch”).

This pattern has worked over and over again in my research group. Here are a few of the problems we encountered along the way and the solutions we generated: “malloc doesn’t scale” (Hoard), “my C program crashes” (DieHard), “I can’t run my C++ code on the web” (Doppio/Browsix), “the profiler isn’t helping me make my program run faster” (Coz), and most recently, “Excel didn’t find my spreadsheet bugs” (ExceLint). (All of these and some recent talks are linked from our lab web page, https://plasma-umass.org.)

Sometimes these problems find you exactly when you are building a system to do something else! For example, the Hoard project happened because I was building a parallelizing compiler for pure LISP. I found that the generated (embarrassingly parallel) programs didn’t scale. The bottleneck turned out to be the memory allocator, which used a single big lock. I went looking for an off-the-shelf solution, or one described by previous papers, and found that they all fell short. The result — after a lot of work — was Hoard. I never did finish that parallelizing compiler :).

Even if solving a problem you’ve experienced doesn’t result in wide adoption, it’s a good thing to do anyway. Applied research should solve clearly articulated problems for which the state of the art falls short. A system that solves such a problem is clearly going to be an easier “sell.”

Build Real Systems

For impact, identifying an important problem is step one, and building a real system that convincingly solves that problem is step two. Aim to produce systems that do more than just barely manage to cross the finish line (read: the paper deadline) and then collapse. Instead, build real systems that people can actually use! Make sure, for example, that your system works not only on every benchmark application from a suite, but also with real applications like browsers or servers. If it doesn’t work with one, or its performance is terrible, fix it! If you have found a problem, someone else will, too. Yes, it takes more time, but it’s worth it. Here are some procedural steps you can take to help make sure this happens.

• Put all of your projects on GitHub or some other public repository, and make everything public as soon as possible. People need to be able to download your systems so they can use them. I personally prefer having the whole process take place in a public repo, but you can also use a policy of “make public on submit”: when you submit a paper, flip the permissions on the repo from private to public. Developing in public (or code that will definitely be made public) encourages everyone to do a better job of building their code, documenting it, and avoid cutting corners.
• Make it clear (to students and collaborators) that the paper doesn’t get submitted until you are satisfied that the system described actually works. Test it out yourself. Kick the wheels. Make sure it works as advertised.

This is a virtuous cycle: once you establish a culture that favors not just doing great research but also building impactful systems, this will attract students who have the same goals, and that means they will build more impactful systems, and so on.

Do it anyway: Building real systems is the ultimate reality check. It answers the question: does your approach actually work? What at first glance appear to be corner cases or implementation issues often uncover deep conceptual problems. It’s hard to think of a single project from our group that did not change significantly in response to issues that arose during the system-building process. In short, our original approaches ran smack into reality and needed rethinking.

Building real systems will also help get your papers accepted, because you can run your stuff with real systems (e.g., running a browser on top of it), and build future research artifacts on top of it, knowing that you have a solid foundation. Incidentally, this will also help your system pass the Artifact Evaluation process; those badges on your paper greatly increase readers’ confidence that your results are real.

In short, building real systems is good for science. It lets researchers reproduce your results, compare their work to yours, or build on it (with the side effect of getting your papers cited). Plus, it can be a huge advantage for students to graduate having built highly visible systems.

Embed Yourself

Embedding yourself in industry is a great way to have impact. Not only does it expose you to more real-world problems, but it offers an opportunity for adoption within that company. I have spent years (literally) as a visiting researcher at Microsoft Research (including my most recent sabbatical). MSR is hardly the only option, though it’s a great one! There is now a huge range of excellent choices of companies you could work with. If you are a beginning faculty member, consider spending a “pre-battical” in industry.

Working at a company is a unique opportunity to have direct impact by influencing (or even developing) that company’s systems. However, this can take time. You probably shouldn’t expect to airdrop in for a month and see magic happen. You need to establish a regular relationship (perhaps including consulting during the year), and ideally, develop a strong working relationship with someone inside that company. For me, that person is my great friend Ben Zorn, whom I have had the privilege of working with pretty much since I was his intern almost twenty years ago (!). Ben has not only been a joy to work with but also insanely effective in getting Microsoft to adopt the fruits of our joint work, directly influencing Microsoft products (first Windows, and now Excel).

Do it anyway: If you are an academic and live primarily off NSF money, you can only charge two months of your summer salary to grants. You can fill in that other month by working in industry.

Collaborating with people is productive and fun! You probably know someone in industry already who you are friends with (or, well, you soon will). The feedback (or even pushback) is super helpful to honing your research and generating new ideas. These relationships also help your students get internships, which is good for everyone.

Visiting a company can also be a great change of pace (and if appropriate, location: summer in Seattle <chef’s kiss>). In particular, while my kids were smaller, some aspects of it were like a family vacation (and Microsoft was generous enough to subsidize housing for my entire family; if this applies to you, ask for that).

Give Great Talks

Talks can be more important than papers. Done right, they can quickly convey why people might want to adopt your research. A great talk will get vastly more views (e.g., on YouTube) than a great paper will get reads. Put in the time to hone your students’ talks (which will eventually become your talks). This is time well spent. Focus on high-level ideas, make heavy use of pictures and animation, and tell a story. Keep formalism and text to a minimum: that’s for your paper, not your talk.

Do it anyway: A great talk will almost certainly inspire people to read your papers. A bad talk, not so much. Training your students to give great talks will also help them get jobs.

Go to the Mountain

The mountain — your potential users — won’t necessarily come to you, so you need to go to the mountain. Academic conferences are mostly filled with other academic researchers: not the best target audience for adoption. Yes, some industry people attend our conferences, but at industrial conferences, they are nearly 100% of the attendees. Give one of those great talks at these conferences. Or give one at different companies. Academics tend to have vastly more experience public speaking than most speakers — we do it at least twice a week — so your talk is likely to be extremely well received.

Folks in industry are looking for solutions to their problems, and by far the best way to let them know about your work is to tell them about it directly. If you give a great talk, it will get lots of views, and you will get invited to give more talks.

In sum: scratch an itch, build real systems, embed yourself, give great talks, and go to the mountain. This advice isn’t guaranteed to get your work adopted, but maybe it will increase the odds of luck being on your side.

Bio: Emery D. Berger is a professor in the College of Information and Computer Sciences at the University of Massachusetts Amherst, where he co-leads the PLASMA research group (https://plasma-umass.org). His research interests span programming languages, systems, security, and human-computer interaction. His group is well known for its influential research and software systems, many of which have enjoyed broad adoption: among others, their work has influenced the development of the Rust and Swift programming languages, and memory managers deployed in both Mac OS X and Microsoft Windows.He also developed and maintains the CSrankings.org website. He was named an ACM Fellow in 2019.

This article was originally posted on the SIGPLAN and SIGARCH blogs.