José Valim: "Elixir is great for everything that runs on top of a socket"

Introduction

Evrone is not just a custom software development company, we also stay highly connected with local and international communities. We organize the annual Ruby Russia conference, a huge event featuring expert guests. These community connections give us a competitive edge and provide us with better technical expertise and consulting opportunities. We have conversations with developers around the world, including the creators of innovative new technologies, and we try to strengthen connections and give back to the community through interviews, conferences, and meetups. We recently conducted an interview with José Valim, the creator of the Elixir programming language, and asked him about his goals for the language and how it came to be.

The Interview

Evrone: Hey José, it’s great to have you here! Let’s start with a small introduction into what you do. After Nubank's acqui-hire of Plataformatec, you started Dashbit. What are you doing right now?

José: Hi, thanks for having me! I am José Valim, the creator of the Elixir programming language. At the beginning of this year, I started a new company called Dashbit. Our goal is to help other companies adopt and run Elixir in production.

Evrone: Awesome! Let’s speak a bit about the popularity of languages and technologies. There are ideas as to why some technologies are popular and some technologies are not. One such idea is that many technologies are championed by so-called "killer apps," like a Ruby on Rails majestic monolith framework for the Ruby programming language. What do you think about the idea of "killer apps" and "killer demos?" How important are "Phoenix" and the "make a Twitter clone in 15 minutes" demos for Elixir?

José: I think they help, but it’s not the determining factor. If you have a killer app that can get a lot of attention in a short period of time, that's obviously going to help with the adoption of the technology, but it's not a requirement per se. Even if you have the killer app, it is not going to be one single thing that can sustain the whole community. I consider Python to be a great example of what I would like a programming language to be, because it can diversify.

It's important for the community to grow beyond that killer app, be able to diversify, and go to different domains. This is the approach I had for Elixir since the beginning. I come from a web development background, and I was very careful to not paint Elixir into a corner, where it becomes a language only for web development. I try to expand the reach of the language. Today, when you look at the applications of Elixir, there are a lot of people using it for the web, but there are also people using it for building distributed systems. This is something that we inherited from Erlang. There are projects like Nerves, which is bringing Elixir to embedded devices in a very exciting way. Projects like Broadway and GenStage that are all about efficient data processing. I am more interested in bringing diversification, instead of trying to find that one killer thing that becomes the main focus.

Evrone: Yeah, sure. Python’s diversity is legendary. With so many programming languages, we developers like to label them. Erlang is labelled as "Telecom language," Clojure is "modern Lisp," and so on. If you had to craft a short label for your Elixir language, which one would you pick?

José: If we were to short label it, I’d say that Elixir is great for everything that runs on top of a socket, like web servers, IoT devices, distributed systems, audio/video streaming, etc. When you're doing anything that relies on the network, if data is going in and out, Elixir will be excellent. So I guess I would short label it as "the language for communication".

Evrone: That's a great selling tagline for software developers! Language popularity is influenced by big companies' adoption. What are Elixir's selling points for such companies? Maybe you can share a success story with us?

José: At the beginning, I knew I would have to go around and talk about the language, go to events, give talks, and write materials. I have been doing that ever since. But when it comes to selling to enterprises, I don't think there's one specific answer. There are different approaches that we can take. I was always lucky to have co-founders and partners who were great at moving sales forward. I come in from the technical side, and then there are people that are business savvy, they bring the business side, and together it’s going to work better. That's why it is important to collect everybody's input. We never try to centralize, because everyone can bring a different perspective.

Evrone: Yeah. Selling things is hard, as is naming things. Actually, we stumbled upon the name origin question a few times on the Elixir forums. Where did the name "Elixir" come from? We found a reference to Potion, the language by _why the lucky stiff. Have you met him? Maybe you can reveal the origin for such an RPG-sounding name?

José: I only knew _why the lucky stiff  from the software he created. I never met him personally, he was active in Ruby before I became active. Well, the name of Elixir is not related to Potion. I liked Elixir because it's short, but not extremely so. It's not used for anything else in programming. There are books, there are perfumes, but nothing technical. Also, it starts with the same letter as Erlang. If there's any kind of alphabetical listing, it's going to come relatively close to the top. It's neat, it's also six letters like Erlang. So I was like, sure, let's go with it.

Evrone: Cool. How would you measure the learning curve for the developers who want to learn Elixir? Is it a suitable language for junior developers who have just one or two years of experience? What do new developers need in order to enjoy the language you created to the fullest?

José: That's a very good question. Some people are able to be productive and grasp the concepts fast, others don't. I personally think Elixir has an easier mental model to grasp than, let's say, object orientation. But people tend to learn and study things in different ways.

I always contrast this with my friend Hugo (Hugo Baraúna). We met at university, and we have two very different, distinct approaches to learning. I am the kind of person that would go into a cave with barely any lights and try to just explore things and see if something lands or not. I am an impatient learner. Hugo is a very structured person. If he was going to explore a cave, he would need a particular lamp and all the right tools first. He would structure everything before taking any steps.

I know there are people that have tried teaching Elixir to people who have no programming experience. And they were successful. There are people coming from JavaScript, Python, Ruby, .NET, Java, who are having tons of fun with it and being productive. Others struggle with it. I’ve heard from some who are trying to learn Elixir for the second or third time, after not making concrete progress the first time around. The second time, it was a little bit better. Then they one last time and everything clicked. 

The good news is: we are at the point where there are a bunch of learning resources in the community. If you are a new programmer, the introductory guides in the Elixir website are likely too fast paced and lacking depth. But we have books, like Introducing Elixir, from O'Reilly, and Joy of Elixir, by Ryan Bigg, that are good for beginners. Programming Elixir is a good resource for people coming from dynamic languages, like Ruby, Python, or JavaScript.

Elixir in Action, by Saša Juric, is about teaching Elixir from a system perspective. It's not about Elixir as a programming language, but Elixir as a tool for building systems. There are also screencasts, online courses, etc.  If you are a person like me, who doesn't have a lot of patience for reading and just wants to try things, there are a bunch of places where you can do exercises. Since we have so many resources, you can try a couple of them out and see what works for you.

Evrone: We know that Elixir was inspired by Ruby, Clojure, Haskell. In hindsight, is there anything else you can name that inspired you?

José: Yeah. The three main ones were Ruby, Erlang, and Clojure. Not only did we get features from those languages, but we also got part of their philosophy. We inherited the whole infrastructure from Erlang. With Clojure, there are a bunch of similar ways we approach problems, like protocols. And Ruby reflects on the syntax, the standard libraries, and how we name things. From those three main languages, we got ideas for features, and also got ideas for the philosophy and what a programming language should be. We got a little bit from everything, from Haskell, Python, F#.

Evrone: Interviewers always ask about the biggest design success and biggest design failure of the language, but according to our interview experience, it's much better to ask just about design success. What is the biggest design success of Elixir?

José: Okay, that's a very good question. When I started Elixir, I built a prototype that was bad in many ways. I was kind of that person lost in the cave, with very little light, throwing things out and seeing what stuck. After it failed, I was able to come up with what I wanted in the language. My starting point was Erlang, and I wanted to build Elixir on top of it.

I had two goals: extensibility and productivity. Productivity was about focusing on having really good tooling. There are other languages, like Ruby, that focus on tooling and the development experience. And a very recent example is Go, which has had a very strong focus on having very good tools since the beginning. Extensibility boiled down to two things. One was the protocols. The problem with protocols is the problem of polymorphism. I've looked at how different languages tried to solve this problem. Clojure solves this problem in a way that would better fit the Erlang Virtual Machine. So I brought that in. 

The other fact of extensibility was metaprogramming. I tried different approaches to having metaprogramming in Elixir. At the beginning, the model was based on Ruby. But Ruby is about dynamically changing things at runtime, and that doesn't work for the Erlang Virtual Machine. After you compile the code, you want that code to be static.

Then I tried Lisp languages, like Clojure, that have metaprogramming based on the abstract syntax tree (AST). I really enjoyed the experience there, but I did not want to implement Lisp. That path has been travelled on and multiple people have already done that for the Erlang VM. I wanted to do something different. So I decided to introduce a higher-level syntax with a straightforward AST representation that developers can use to metaprogram. And for a while, I thought that was my biggest design success!

However, later onI learned that John McCarty, who was the inventor of Lisp, also thought about that. He introduced SLisp, which is what everyone calls Lisp today. And he also introduced the MLisp, a higher-level syntax. So my idea was actually a 50-year-old idea, but I think Elixir is one of the few languages that is using it in practice.

So I guess we can say my biggest design success was looking around and seeing what made sense to be brought to the Erlang VM and in the most cohesive way.

Evrone: Elixirforum.com is very popular. From your point of view, how unique is this resource to the Elixir community and does it make a difference to have your own community platform, compared to other languages and ecosystems which use mailing lists, groups, Reddit, and StackOverflow? How was it created? How is it important to language development?

José:  I've always tried to make Elixir a community effort. I did not try to centralize it, I wanted to give everybody space or opportunity to make a difference, do their own contributions, and get involved in whatever way they wanted. We have very few official spaces that are controlled by the Elixir team. So we have always been very welcoming of people who wanted to create their own spaces.

There are even the language barriers. I know there's a big Brazilian Elixir telegram community, for instance. Therefore it's more important to allow those places to grow on their own and do it in a healthy way that is consistent with the standards and the goals we have for the community.

With the Elixir forum, it was similar. Aston created the forum, and it's great to have a new space. He is the one who pushed it to grow and to get where it is today. It's one of the places that I enter from time to time to see if there is something that I can help with. But, you know, as far as it goes, it's one community place and there are many others.

Evrone: It's amazing. What IDE would you recommend for Elixir developers?

José: Well, I don't focus a lot on my editor. I used TextMate 1.0 for probably a decade, even after it was no longer maintained. I only stopped using it because the new operating system no longer supported it. Then I migrated to Sublime Text, but I don't use many of the editor features. I have a hotkey to format the code, but that's pretty much it. I know a lot of people are using the Visual Studio Code because of the Elixir Language Server. There are great developer that are working on the Language Server and making the integration better and better every day. So it seems to be kind of a general preference.

Evrone: Amazing. The next question is technical. What do you think about the new "async" paradigm vs the "message passing?" Which is better for a developer's brain to get around, excluding performance and stability issues?

José: The thing about async/await is that in different languages they're implemented in different ways. Some are implementing async/await because they want to provide I/O based concurrency. Others are not and we need to make this distinction. So let's take a step back.

When we are talking about our software, there are two types of concurrency that you want to think about: CPU-based and I/O based concurrency. Languages like Erlang and Go show that you don't need async/await to have I/O based concurrency. It's completely unnecessary. From that perspective, I don't like the async/await model when it's used to provide the I/O based concurrency, because it's not needed. This is pushing the I/O based concern to my software and that’s making my life harder, instead of easier. Because that's something that the language should have solved, and it should not be pushing me, the developer, to solve it. On the other hand, Elixir does have async/await, but it's just a convenience, it's about CPU-based concurrency.

I want to expand on this more because this is the reason why I got excited about Erlang. The reason I fell in love with Erlang is because, while everybody was trying to solve concurrency, Erlang uses the same abstraction to provide concurrency and also distribution. I can have two entities running on different machines, and they can communicate with each other. This same abstraction also provides a model for you to think about failures and fault tolerance. When it comes to I/O, you don't have to write callbacks or invert the control-flow, the runtime takes care of it for you.

That's what is really exciting about Erlang. It's an extension of the actor model. It’s the actor model plus fault tolerance with it. It's one thing that encapsulates many things and this means to me that, as a developer, I have to keep less things in my head. I don't have to learn a bunch of different tools or a bunch of different models that apply to one specific concept. I have one abstraction that controls all of this, and I find that very elegant.

Evrone: What is the best-case scenario for the future of Elixir? Where do you see the language in 10 years, if everything goes (extremely) well?

José: I’ve never really thought much about it. Starting a programming language is such a big effort and most of those efforts "fail". I knew it would be a huge investment of time and money, in some sense, as well. I did not want to build a huge expectation towards it because, if it didn't work, I would be like “oh it failed, it’s normal, it's expected.” So that's what I did in the beginning. But I still wanted to do my best. I wanted to have a diverse language that is not dependent on one killer app, but has a bunch of different domains and areas that the language can work with. And I wanted to have a welcoming community.

After all of these years, my long-term plan is pretty much the same. And I am still interested in seeing how the language can diversify into new domains. 

For example, there's a bunch of things happening with Machine Learning (ML), and it would be awesome if Elixir became a part of that. In ML, when you want to recognize if something is a dog or a cat, it's not like a single question-answer like, “oh this is a dog or this is a cat.” You break it into many properties. Does it have cat ears? Does it have a dog nose? So what you actually need to do is to have distributed learning, with different nodes you want to be focusing on specific features, and at the end, come to one specific conclusion. 

This would just be fantastic for Elixir. Because this kind of distributed, concurrent learning, where you have to coordinate a lot of things happening at the same time, it's something that the language is really good at.

Evrone: If you could travel back in time and give only one piece of technical advice to your younger self when you started to develop Elixir, what advice would it be?

José: I don't know, to be honest. It's not like I don't have any advice, but time travel is very complicated. What if I do the time travel, and then I give the advice and I accidentally teach my younger self to screw things up? How can you be sure that if you go to the past and you change it, the outcome is going to be better or worse? Maybe some things have to be learned the hard way.

Evrone: True. What languages do you consider to be Elixir competitors and why?

José: Generally speaking, I would say other dynamic languages. But if we go into each particular domain where Elixir is used, you may have a different competitor. If we speak about the web, it's obviously going to be Javascript, Ruby. If we're going to data processing, it's more likely that we are competing with JVM. But in general I would say the other dynamic languages, because, given their properties, dynamic languages tend to feel similar. 

Evrone: You work a lot on the language, it's ecosystem and community. What helps you to maintain a work-life balance and not burn out?

José: There are many aspects to it. My number one advice is to shut everything off, as if your computer is always in do not disturb mode.  That’s the way you can focus and have some sanity. The consequence of working from home is that people are going remote and depending more on the online form of communication. I am very proactive at cutting down things that require my attention. If there is important information, it's probably going to get to me in some other way, and it will be fine.

For example in the past, I was very active on Twitter, but I later decided to cut Twitter from my life. I used to follow a lot of people who brought me knowledge because they would tweet about new things. I was worried that, once I did that, I would no longer get that information, but that was fine. 

I have been working remotely for more than a decade. When 2020 happened, not a lot changed for me. I don't have those things on my phone either, because otherwise, it's really easy for you to get distracted. You go out with your friends and then the notification pops up and you want to look at it. I try to not have things that distract me or are going to try to take my attention from what I'm doing or what I'm supposed to be doing at the moment.

Evrone: It's awesome advice! It was a pleasure to speak with you. Thank you!

The conclusion

We had a great time speaking with José and learning more about his approach to building Elixir and how he developed the language. Also, we'd like to thank the Ruby Conf Taiwan for the picture used in this artcile. Here at Evrone, we frequently use Elixir to create custom solutions that meet our clients’ unique needs. We love it when we get an opportunity to learn from a technology’s creator because it strengthens our expertise and gives us more tools that we can use to develop innovative new products. If you have an idea for a project that would benefit from Elixir, we would love to help. Let us know how we can contact you, and we’ll be in touch soon to discuss your idea and how we can help you make it a reality.