Inquiry into knowledge & tech
A conversation with Michelle Williams, the ignitED teacher about technology adoption in schools and how digital devices in the classroom are essential to close achievement gaps.
Core concepts we discussed:
During this in-depth conversation with Dr. David Grew, a radiologist and physician, we explore how the innovative art of visual storytelling can support communication between medical professionals and their patients.
Core concepts we discussed:
And here’s the map of various concepts we flagged during our conversation. Use controls in the lower-right corner to zoom-in.
Exploration of the science behind how we learn. A conversation with Reshan Richards, educator, writer and co-founder of Explain Everything whiteboard.
We humans are storytellers. It’s the stories that carry ideas no matter what technology we support ourselves in the process. In the conversation we look into five pieces of scientific research to explore correlation between the nature of representation and clarity.
B.A. Gonczarek
I’m here with Reshan Richards, welcome Reshan.
Reshan is a researcher, educator, and a writer of two books on technology and leadership, with the last book titled Make Yourself Clear on how to use a teaching mindset to be understood. Reshan also co-founded a whiteboarding platform used for teaching in many schools across the glob and it’s where we teamed up. And this cross-roads of technology and education is something that I’m sure will be inspiring for our listeners when we discuss the science of how we learn. Was my introduction of your background accurate Reshan?
B.A. Gonczarek
I’m here with Reshan Richards welcome Reshan.Reshan is a researcher, educator, and a writer of two books on technology and leadership, with the last book titled Make Yourself Clear on how to use a teaching mindset to be understood. Reshan also co-founded a whiteboarding platform used for teaching in many schools across the glob and it’s where we teamed up. And this cross-roads of technology and education is something that I’m sure will be inspiring for our listeners when we discuss the science of how we learn. Was my introduction of your background accurate Reshan?
Reshan Richards
Oh, I think so. It was a very kind and generous introduction. Thank you.
B.A. Gonczarek
Great. So, to begin, let me confront you with this thought. We humans are storytellers no matter what technology we use in the process. The stories are very efficient keras for ideas, would you agree?
Reshan Richards
I would absolutely agree with that statement.
B.A. Gonczarek
There are two interesting avenues to explore: On the receiver’s end – there’s the question of “how we learn”, on the presenter’s end – how do we provide stories? The right combination of both is essential in education, to learn things, it is also essential in persuasion outside educational realm. The better we tell our stories the more we improve chances for being understood. Right?
So my goal for today is then to explore with you the scientific foundations of visual storytelling. Let’s try to provide for the benefit of our listeners. What are the underlying facts of storytelling, we gathered scientific papers were inspired with and we’ll use those for our discussions will be doodling and sketching while exploring the papers. So if any listener would want to see us doing that, please use YouTube link provided with the podcast.
Now we’re Reshan, we have five stunning articles to discuss and I’m looking for a good starting points, maybe let’s cover the difference between the verbal and textual first.
So, this one comes from knowledge Media Research Center in Germany, and the article asks what improves learning results, text or pictures, while the bulk of existing research is focused on the sequence in which text or picture is provided in learning. This paper, however, operates under the assumption that it is not the sequence rather the function of text or picture in the process of learning. So they look at the function and they defined text. It’s actually in this part here. Then define text as verbal coat in a short or long prose Or instructions, let’s say. However, pictures come in different forms as less or more abstract representations of objects that still contain some similarities. So pictures in the research can be a photo, but also a doodle, or a map diagram, graphic organizer, or let’s say concept map. And the conclusion the way I see it is that the type of information to be learned dictates the way it should be presented. Basically, some materials are better one learned when provided us text other if they come in form of a picture, or they also suggest that the complexity of information is the key. And they conclude, acknowledging that their work is more of a guideline for future research server. Reshan, what’s your perspective as a teacher and a graphical facilitator at the same time, what your experience tells you about using pictures versus text?
Reshan Richards
Yeah, it’s a great question. I think it’s a great kind of area of thought. And I’m chuckling as I’m looking at the closing statement from this research, because to degree, almost all new contemporary research on emerging phenomenon always include this idea that hey, well, this should now be extended into a further study, which it’s, it’s so true, right? Like not, nothing is conclusive yet. And I think what’s also interesting about what you shared in thinking about the, you know, that it’s about choice, right? So if you’ve got an idea or concept or some bit of knowledge or understanding that you’re trying to convey or build in somebody, you’re certainly going to make choices about, you know, either the sequencing or even the more binary choice of text or image as like, well what’s the best medium to convey that, but I think, you know, that that’s still to a degree one directional so as a teacher The other thing that we’re also going to be thinking about is about the receiver. And the reality and kind of the emerging research that validates it that people have different modalities that depending on the concept or content that they might be more dominant in. So for example, you know, there might be some people who for certain types of topics or disciplines that are much better auditory listener, so we’re and then some people, while they might be highly visual, they might be visual towards text and reading narrative. And another visual learner might also be much more into photo and imagery. So ultimately, as as people who are trying to communicate to teach trying to persuade, as you mentioned before, there’s just it starts with a baseline understanding that you do have to be really intentional about the format that you’re presenting the content information in. And then you also need to be really thoughtful about the learning styles and channels of that audience member.
B.A. Gonczarek
Yeah, yeah, absolutely. So, you know, when looking from my perspective, when I learned complex things, I like picture to form a structure and then leave the details to sit inside of a picture inside of picture as a text. So it’s essentially the same idea as infographics, where we, we mix graphical form with textual content. So this “blending” thing is it’s something that there is no clean part between one modality and the other that you suggested is rather at blend to convey an idea, isn’t it?
Reshan Richards
Yeah, that’s a great way to describe it. Blended media, mixed media, mixed modalities. I think anybody who’s interested in, you know, the types of things we’re talking around here. Those would be those would be good. words to describe it like and, you know, yeah, this blend this mix of, of a representation or presentation of information.
B.A. Gonczarek
Right. So let me present you a second piece of research that discusses the suitability of medium again, pictures of loads when communicating with either people we know, or strangers. It turns out that we’re inclined to use different medium. When sharing thoughts with someone that is known to us and different when we communicate with someone we don’t have much in common. So the authors here, they describe six experiments to build the evidence that it is, in fact, the distance that sets our preference for choosing between using text and pictures. And again, in this paper as before authors looking to unique characteristics of both pictures and text. Pictures are defined as concrete representation as analogies of the real world. While words nearly always are abstract, with arbitrary relationships to their corresponding subjects. They even stayed that the word is actually a category that refers to a broad range of concrete objects. Words carry the essence of an object, but usually not its properties. So in result, it turns out that pictures are more often use among friends. And the use of words translated better over distance. What’s your reaction to that?
Reshan Richards
That’s so interesting, because I would have expected it to be the inverse and mostly because when trying to have the challenges of distance and proximity or lack of process activity, you would think that the more concrete representation would be helpful for kind of eliminating misunderstanding. So this is really interesting that it’s actually, at least for people’s preferences, what they measured in practice, that those who were working with known people tended to use more images. But part of that also, I’m so curious that like, in their work, is it also that familiarity means that in one ways, you’re being concrete but you’re also not doing some of the formalities as far as supplying kind of preamble or context for the thing that is sent. Whereas with a stranger or across distance, you almost do have to use more words to kind of set up somebody for whatever is trying to be communicated. I also thought about you know, I’ve done a lot of work and teaching across distance and having meetings and conversations and you know, small and large video conference. rooms. And I have found that those types of meetings are always more effective when either there’s an existing relationship that you have had some closer in person time with the parties so that your first meeting isn’t the one that’s there, or if it’s going to be like a regular ongoing thing that you you find a way to blend it together. And I what I mean by this is, you know, a lot of companies are certainly looking to do more sales motions, more customer service, customer success types of things online, but I almost I think it becomes a mistake based on like, some of the things you’re describing here to think it can be 100% substitute for that real kind of close on connected engagement. And I think the companies who are succeeding are realizing you can be more selective about when you do have those in person content. But then you can still do a lot of the other work using digital media light, like, you know, video conferences or what we’re doing a collaborative whiteboard to still like maintain and strengthen that relationship.
B.A. Gonczarek
Yeah, absolutely. I think in that setting, you would want to find a solution to close the gap to get you closer to the to the recipient of your information. And I agree that the findings here are kind of counterintuitive, but there’s a good reason why is it so. The authors refer to CLT (Construal Level Theory) that states that people increasingly use more abstract representations when communicating with those they don’t know that match. And the reason for that is that people seem to be afraid to provide too much personal information to those that are not yet considered friends. Therefore, they use disembodied linguistic representations of text while leaving concrete contextualized pictures for communicating with those that they have already. something in common? You see the point here?
Reshan Richards
Oh, absolutely. Yeah. I mean, when you describe it that way like I can, I can fully relate that there’s this, this the psychological element in in detaching any potential clues or to specific or to accurate details because of not being too familiar with the audience. So that’s that that makes sense, even though as you said it on the surface feels very counterintuitive.
B.A. Gonczarek
Yeah, absolutely. The visual representations seem to be more appropriate when we already share a piece of reality with the person on the other end. That’s, that’s basically what they found. But I wonder, in teaching environment, I guess being specific and being clear, is the goal, right?
Reshan Richards
Oh, absolutely. I mean, that that’s the thing, can you not only you know, transmit or share some idea or concept but clarity comes from confirming understanding, and that with the most efficacy or efficiency, that that understanding was built. I mean, that’s really what what clarity happens, like I have something that I want to convey or to build in somebody else. And clarity comes when that is done with the least amount of friction.
B.A. Gonczarek
Right. But does that actually says the preference for using pictures instead of woods?
Reshan Richards
I think that anytime you can be more concrete… Let me let me rewind that. I do believe there could be a correlation between the concrete nature of a representation and clarity. I think so. So here’s, here’s an example where the mix of pictures and words in a professional setting can can often get to get in the way of that clarity. So often when doing kind of employee or, you know, supervisor evaluations and review comments are often left in text, right? So, a supervisor might write text comments typed up in a narrative added to a file, dot, dot dot. And so it’s immediately already a couple of degrees away from the actuality of the work, right? Because it’s the supervisors interpretation of what might have been observed and communicated, but then re re put out in text form. Whereas if somebody just had a simple snapshot, or a video of that behavior, action, or whatever somebody being, their performance is being evaluated in you almost removing all of that abstraction. It’s like super concrete, like, how does someone so lead a small group meeting? You know, I could write a narrative about it. Or I could just make a 32nd clip of here’s how somebody kicked off their meeting. And that concrete thing will actually get to that greater clarity of the situation. But I still as a supervisor, would want to be able to comment What’s interesting or important about that thing, but if I’m both describing it and then commenting on it, it’s immediately a few degrees of separated away from what you might have been trying to look at in the first place.
B.A. Gonczarek
I think that’s a, that’s a great example. So moving on to the two pieces of research that you provided it on memory and meaning making, which one you think should come first, which one would you like to tackle?
Reshan Richards
So I find that that this picture of superiority effect body of research is really important because I want I think it’s connected to the two studies or articles which you referred to, and what I like and it took so first of all, it’s already building off of previous research that to a degree validated, you know, in in pretty controlled experimental studies where somebody was given just an audio representation of some Something, something that had audio plus text, and then something that had audio plus image in these controlled experiments, the retention. So this is really about like recall and memory call. But the audio plus picture almost always outperformed any of the other two groups, right as far as people’s retention and recall. So that’s interesting, right as a baseline, but you can’t make too many generalized conclusions just based off of you know, these kinds of experimental studies. This particular one that I selected and pulled out that builds deeper into it was looking between correlate or looking for correlations between the sorry, it was looking for correlations of the extent of this effect and age. So as you get older, how much more pronounced is this picture superiority effect over the other two modes. And this particular study did find that there is absolutely a correlation, we’re not going to call it causality. But as people get older, this picture superiority effect actually escalates and gets reinforced even stronger. And it’s because when you add your days and months and years of life experience, you’ve had time to reinforce these mental images or mental models or visualizations of different things that you’re capable of having certain retention and recall in a greater degree because of all your prior knowledge. So while it was still present in younger people, it was the the difference of the the extent of this effect was far less pronounced than in their older subjects.
B.A. Gonczarek
You know, that’s quite fascinating, you know, it also proves how much we learn about the picture superiority effect recently. You know, the last time I checked her there were like 40 articles on the subject and the number is growing rapidly these days, too. So it shows that this area still needs further inquiry to what I personally like about picture superiority effect is its linkage with our spatial memory where location of elements help understanding and I was taking by illustration explaining the historical logic using just one picture I don’t know if you if you know that one.
Reshan Richards
Now tell me more about that.
B.A. Gonczarek
It’s the picture that describes the entire logic of Aristotle. Once you familiarize with it, the beauty of that is that you can literally close your eyes and the recall by location each bit. It’s a breathtaking example of how of how this effect works yet, we don’t know for sure still, why it works and in what age group would work the best. Let the quickly pull the example of this illustration real quick to our whiteboard here.
Reshan Richards
Yeah, I’d love to see that. Yeah.
B.A. Gonczarek
Okay. That was something that was used for teaching students a long time ago, though, but it’s still a real life example of using picture superiority effect. I don’t know for which age group, but still, I still love more of those visuals and photographing being used in schools for stimulating retention and recall of information. I’m sure you’re on the same with me on that.
Reshan Richards
Yeah. This is a really cool example here.
B.A. Gonczarek
And the other material that you provided to the canvas was on doodling, right?
Reshan Richards
Yeah, so this one is connected and it’s actually a little bit more about both the the presenter, but also the idea of inviting other to kind of doodle and visualize along with you. Now in this study, which also again, it, it has its limitations because of its experimental nature. But it was, it was having people listen to the audio, and that the group one was, as they were listening, they weren’t doing anything else. And then they had to do kind of like a rent attention and recall. And the other group was doing, as they were listening was also just asked to draw and doodle on whatever like notepad they have. And somehow, the kind of externalization of, of just kind of like sketching, jotting things down I believe some of their notes were relevant. Some of them were just kind of like random, like the things that you might see in a in a middle school, child’s notebook. But there was still a measurable effect on retaining When other kind of expressive channels were active at the same time that somebody was hearing some audio. And what’s interesting to me and the things that I like to dive into is that this in this study, there was no real guidelines around the doodling. But in thinking about situations where you’re trying to build understanding and somebody and you presented some information, having them activate more channels, and then just their listening and their visual, cognitive channels, but actually starting to a little bit of like haptic or even kinesthetic action, and at the same time and this so I put this notice here about random versus purposeful doodling that I think there’s so much to be learned and considered that when you’ve got people engaged, that you’re trying to trying to help them learn. Have them activate the same positive principles around visual visual storytelling in their own note taking and reception and consumption. The information right so it shouldn’t all those benefits that we’ve kind of highlighted and talked about from the delivery standpoint to me have just as much value in the recipient or participant and I felt like this study in particular is kind of a stepping stone or starting point in considering how the recipients channels can be activated towards building that greater clarity and understanding
B.A. Gonczarek
Right, but you know, for me, this shape-shading task also reveals something that is counter intuitive, you know. How come one might recall better if the thing he or she is drawing is not directly connected to the subject of the discussion, right? I find it surprising and I would wish to know more about what was at work on the cognitive level.
Reshan Richards
Yeah, and you know, like for me as somebody who you know, does practice graphic facilitation or sketch noting or however, visual note taking all these different terms that it can be described it I do find sometimes that even when I’m trying to be a very active listener, that I will do some random doodles or shapes, kind of in the meat in the midst, or in the middle of maybe some things that were a little bit more purposeful or relevant. And sometimes they’re just these kind of, in between or bridge moments between like, big concepts. And it’s almost this like, continuation instead of like being like, on off on off. You’re just constantly like in a flow. And yeah, sometimes they just become things of the background or filling the page. But in, in your, in my own review of the things that I might have taken notes on. I remember when I see like a random structure like this, oh, that was an in between moment. And I don’t have to admit, I’ve actually visualize something that I know I don’t have to worry about later, which is kind of almost like how often do you take notes on things you don’t want to remember? But but it brings back recognition of the like the setting or the scenario. So like, I don’t know, in some weird way, maybe maybe this is like the words of a maniac, but like I actually find it helpful to have useless notes, but be able and like they actually helped me filter out the things that are more meaningful on that same page.
B.A. Gonczarek
Right. So actually font something that is related to this, but actually speaks about drawing representations of concrete scientific phenomena instead of just, you know, shading something or, as you mentioned, doing useless drawings. This is an inquiry wood drawing does when we learn science, and it’s an article that comes from August 2011 issue of the Science Magazine, and the authors here… they make a point that visuals are crucial for learning. They suggest that the proficiency in science requires learning to develop representational skills. And I don’t know about you but I’m easily persuaded by that as I cannot imagine how anyone could cope with, let’s say, a concept of particles or chemical reactions without abstract visual representations, right?
Reshan Richards
I mean, what’s so interesting is I think we could probably like do a little bit of like historical digging that any kind of scientific phenomenon or principles, it was probably easier to pass those down in Britain, because of like the printing press, the printing press can much more easily replicate, you know, the written or the printed form word than it could have with graphics, right? Because you didn’t have type sets to pull from to create, you know, highly complex visual drawing. So, you know, the original researchers and publishers from back in the day probably had to default to very text heavy ways of transform, transferring and scaling out information and then it just became a default. I think maybe only in recent times has the technology. And the formats of publishing become a little bit more accessible to let’s call, let’s call them lay people that, you know, these visual representations as a dominant form as far as impacting science proficiency, like you said, I mean, seems obvious, like, of course, how could it not?
B.A. Gonczarek
Yeah. I also remember that from from Dürer’s perspective, Albrecht Dürer’s. During his times, were the times when the printing press was brand new, and he was already trying to find a way to incorporate those hand drawings into the printed materials just to inspire thinking, you know, to inspire this mode of exploration. What is interesting in this article, though, is that it mentions the surveys that indicate that those students that drew to explore to coordinate or to justify understanding where more motivated to learn in the end. So basically we can improve understanding, in other words, by working on graphical representations of what we start to know or what we already know. So that’s something that I actually really like – stating that the drawing is in fact, way of reasoning. And it’s a different kind of reasoning than, let’s say, argumentation. It’s the creative reasoning to engage and explore ideas by refining your drawings of concepts. So instead of looking at the picture, as some sort of fixed universal illustration, we begin to see it as an artifact of cognition. So I’m sure Michelle, that this is essential from the perspective of a teacher, isn’t it? What are the opportunities for having such a window into students thinking?
Reshan Richards
So the thing that was resonating or stirring curiosity in me as you were sharing that were these connections between drawing As a form of reasoning, but the the embedded layer which I jotted down in my margins here is around intrinsic motivation. And I think through reasoning via drawing, I think I would surmise that the reason that students might be able to perform or share or achieve greater proficiency might be because the process of drawing as a form of reasoning is like an intrinsically motivating exercise. There’s a reward in being able to communicate and demonstrate understanding and kind of untangle complex things. That’s kind of harder to do in a more abstract setting, like trying to use the constraints of text and paragraph and sentence structure and all of the other elements that are part of printed text that again are separating the learner from you know, The true expression or representation of what is being thought. And it doesn’t surprise me. And I absolutely think there’s connections to what we’re trying to as what as teachers were trying to achieve in classrooms, which is to instill this kind of love or intrinsically motivated joy around learning, so that it’s not seen as transactional, but rather, it’s it’s fundamental. It’s pure and natural curiosity.
B.A. Gonczarek
Yeah. Yeah. And what is also astonishing that drawings actually lay a foundation for future learning. So it helps students to discern key features and also understand challenges as they go. Right.
Reshan Richards
Yeah. And I mean, there’s, there’s some like really interesting, like papers and articles that talk about how drawing traditionally in schools and I’m talking about K to 12 schools, and I mean, certainly even higher ed, that there’s a perception that Using imagery using doodle sketching things out, is childish or juvenile and not academic or scholarly enough. I think it’s exciting because of conversations like these and pulling scholarly research and really trying to surface all of the values and true like cognitive benefits of visualization and doodling that it might contribute to changing that perception that, hey, I can write my dissertation in the form of a graphic novel, right? And somebody has done that before. And you and I have talked about some somebody who went to who did their doctorate at Teachers College at Columbia University, in that form. Now granted, that person may have been purposefully trying to brush up against norms, but it was completely valid and you want you need kind of pioneers and people to do things like that. To set the example like there are other ways to convey excellence to convey scholarly work to convey findings from research that don’t have to fall into kind of these traditional formats. I mean, all of the articles that were pulled up here, if you look at them, they look like very traditional papers, right? Because that is kind of the societal norm for conveyance of knowledge. So that almost brushes up against what we’re talking about here, which are the many other ways that people are able to transfer into to receive information.
B.A. Gonczarek
Yeah, absolutely. I think you’re spot on here Reshan. Before we close. Let’s try to briefly summarize what we’ve learned from those papers that we discussed. I can try to do that unless you’d like to start first.
Reshan Richards
All right, let me try to zoom out on the big picture here. Okay. And so, let me try to think of like three overarching cues, thoughts that that anybody who’s engaging in this, this conversation might take away. So the first is, there is a growing body of contemporary research. that’s helping make clear the links between visualizing information and better proficiency and understanding it exists and it’s increasingly valid. Second would be that people who have the job or charge the responsibility of communicating to others have to consider the ways that people learn and receive information in their design. There’s never going to be one strict way. But there’s going to be many modalities and modes and today there are far more tools and choices for being able to do that kind of communication. And the third one would be around kind of challenging traditional constraints and norms around what professional what academic communication is supposed to look like, when in the end. It’s really about being clear and getting to a point of under Standing as effectively as possible.
B.A. Gonczarek
Well, I think I fully agree with you. One additional thought to the second point that you mentioned. For me, I’m most excited about this concept of drawing as a form of reasoning. And the reason I’m excited about is that because technologies now are so good at broadening concept of growing with, let’s say, motion or ad-hoc provided animations. The technology makes it easy to transform existing digital assets into something I like to call “smashing bits & pixels” to represent what you need from what you have in your documents or photo library. So a collage, a digital collage is one example of that, but also, our use of the whiteboard here during this conversation is another example. I just think that the phenomenon of Visual Thinking is empowered by the use of digital medium. So before to be a digital thinker, one needed to acquire at least basic drawing skills, right? But now with digital tools that we have, that seems completely optional, as instead of drawing, one can rearrange existing representations to perform, as we call it here, creative reasoning, and be open for all benefits that we discuss here. Would you agree?
Reshan Richards
I agree. Absolutely. And I would say that so many people, when they think about having these kinds of open ended visual experiences, say, Well, I can’t draw so I can’t participate, but all you need to everybody, if they’ve got, you know, you know, their, their, their hands and arms or whatever else they can manipulate to create something on a piece of paper or on a screen, can draw a line can draw a circle can draw an arrow. These are not advanced, you know, you don’t have to go to art school to be able to construct those things. And because as you said, the technology helps make it easier to bring in different types of visual content. This isn’t about being a great artist. This is about putting visual information together in order to help things make more sense.
B.A. Gonczarek
Well, Reshan, thank you for sharing your perspective. I’m positive that are many ways we could further explore the scientific foundations of learning and visual storytelling. So let’s pause here for now here the reaction of our listeners and perhaps continue in the future, shall we?
Reshan Richards
That sounds great. Thanks very much.
Here are the resources we discussed during our conversation:
A conversation between Thomas Nail, an Associate Professor of Philosophy at the University of Denver and the author of the recently published book “Theory of the Image,” and B.A. Gonczarek, your host.
A philosopher’s perspective on the nature of digital images, their material roots, and various consequences which escape our consciousness. Why the digital is more analog and material than we think and how the origins of this revelation go back to Rome. How viewing a painting makes us a part of it? An attempt to explain communication on a more fundamental level than the cognitive. How we’re progressing with the development of technology, how new frameworks can support our understanding, and how we continue to risk missing the point with existing frameworks.
B.A. Gonczarek
I’m here with Thomas Nail, the Associate Professor of Philosophy at the University of Denver and author of recently published book Theory of the Image, welcome Thomas. I must admit I was really looking forward to our discussion. When preparing to our conversation I did my research online and I was taken by how well you’re received by your students. You students describe you as very knowledgeable and approachable. Your openness is something I experienced myself, so thank you for the opportunity of doing this podcast together. And to explain to our listeners – what we’re trying to do here is to (possibly) bridge the gap between abstract thinking and acting, between thought and execution by an exchange between you, as a philosopher and me, as digital toolmaker on a topic of digital image.
My main goal for today is to hear your point of view on the future and possibilities that technology gradually unlocks. Now, I’m aware that the digital image is only a short chapter of your recent publication but I believe that limited scope of our discussion is enough to inspire our listeners. After all, we’re all users of digital devices don’t we.
To begin, describe to us, if you will, your way of working. What is New Realism and what is your method of approaching problems?
Thomas Nail
If I had to sum up main findings of the book that guides the whole project is that the image we often think about as a mental representation, something as in our brain (in our minds) which is a copy or resemblance of the world outside. I think that’s not right, there’s definitely something going on but that’s a very narrow way of thinking about what an image is.
An image is a real thing, it is something that happens in our eyes and in our brains, that is related to the external world, but that is a tip of an enormous iceberg. That’s the part that we see on the surface. Below the surface of the water is this enormous process of the rest of the world, of the enormous processes that we don’t actually see which are part of the fabric of the world and forms and media that we use, and it’s very active. What we have in our brains is not a copy of the world, it is the world itself just by other means. It is a continuation of the world inside of us just. It’s not a question of resemblance but interactivity, of performativity. We are interacting with the world when we see, although we often experience vision as a passive thing that sort of happen to us, but that’s actually very active both in our bodies (in our eyes the way they seek out, move and follow and respond to the world). One of the main takeaways was to think about much larger context what an image is but also what the world does. Whether are humans there, or not, there are images, as they sort of they engage each other. The way we interact with the world those images interact with themselves and that interaction is what produces an image. That’s a broad definition of an image but the shift is from thinking about images as representation to thinking about images as processes with their own habits, cycles, they sort of interate and respond to each other to produce meta-stable states. They are flowing and moving, but they are also stabilized, so they look static. If you look at an object on a table – it looks like it is just sitting there but it’s not. And even when we weren’t looking at it, the image is still because the image is real and material whenever we think about that. We’re part of it when we view it.
B.A. Gonczarek
As you describe in the book the image is a process by which matter twists, folds, bends and reflects itself into sensations and affections. What was the inspiration to arrive at such viewpoint on the image itself?
Thomas Nail
It’s an old inspiration actually, it goes back to the Rome and poet Lucretious. We only have one book of his philosophical poetry – De Rerum Natura. In the book, inspired by Epicurus, who said that the earliest Theory of the image as a material process that we have in the history of the west and it’s since been transformed by other ideas, but I do think there’s something to go back to. For me the inspiration was his poetry and ‘Simulacra’ – Everything in the world is radiating out images. Images are bouncing off each other, eventually they get to the exterior and fly off to collide mid air with other images. Some people interpret it as there’s ghosts flying off of things, but that’s not what he says at all, it’s actually closer to modern physics and light. He didn’t use the language of photons, he used language of simulacra but that’s essentially what it is – that things inside of themselves are vibrating with photons. Photons are heat, photons are light, they are constantly vibrating and release waves of photons, and photons collide in mid-air. And for that reason at every stage they actually are making something, they performing and producing. There’s no resemblance, but no genuine copies, no originals, there’s just these singular processes that refract (like you drop two paddles into a pond and the ripples would key each other and make a new pattern – at every stage you’re always looking at some specific pattern of the photons interact with each other. So it’s a very materialist way of thinking about what an image is as opposed to the idealist way, which is – it’s an idea I have in my brain. And if that’s what you think an image is then only humans have them, only humans can sort of talk about them and they always will fail in representing the original image. There will always be some poorly construed copy of what’s out there. If you think of an image of a real, material, singular process, then it changes the way you think about what an image is. What an image is is what an image does. It doesn’t represent anything, it moves, it does. So the question is – what are the patterns? That’s why I think the visual aspect makes a lot of sense because to understand what images are you need to have an interactive and visual tools to map out what that image is doing.
B.A. Gonczarek
And I believe that’s also applicable to the digital world. I found it actually fascinating of how you’re shifting perspective here. You name three features of mobile nature of the image and I’d like to ask you about hybridity that you list as one of those defining features. You call it a pinnacle of fragmentation, I’m curious what opportunities fragmentation opens`?
Thomas Nail
When I say fragmentation I don’t necessarily mean complete isolated fragments. They are little knots and pieces of strings, always related and connected with another pieces so that the pieces are never fully cut off from one another. This is the way people tend to think about digitally as just fragmented bits and bites, ones and zeros – but there are no fragmented ones and zeros that are fully cut off. That sounds opposite to the definition of what we think of binary. The truth is if you just dig below that level – is a signal on or of (basis of digital communication) and look at the material structure of transistor – it doesn’t work like that. There is a constant flow of electrons and photons moving through that transistor and they do not always stop at the gate when the signal is supposed to be off – they jump the gate. It’s a quantum effect called tunneling in which electron movement actually passes the barrier. The smaller technology gets, the more data we can store, the tinier the gates get. And the tinier the gates get the easier it is for the flor of electrons to pass through the gate and then you get an error, and your computer crashes. And these are happening more often than used to because of the technology. When your computer crashes there’s a good chance that’s because of the quantum effects of the material movement of the electrons. So thinking about all these pieces it really draws your attention to the creativity and the agency of the matter itself that we’re dealing with. We try to represent things of ones and zeros but what we’re often encountering is this very fascinating resistance of the matter itself and that opens new possibilities of working with that matter as opposed to trying to dominate it and trying to stick it into a binary code. Oneinteresting question for the digital age and XXI century is what new things might we discover? What new visual or communication aspects if we let the materiality, if you will, to play a role and speak instead of trying to silence it or make it your bidding. What might it say to us? How might we use it by working with it as supposed to trying to master it.
B.A. Gonczarek
Absolutely. I remember from your book when speaking of hybridity you touch the digital foundation of the image saying that
“Anything that can be coded can be transcoded and then turned into a hybrid of something else.” So the beauty of transformation and allowing for new thing to arise from something that preexisted before opens a lot of new possibilities.
The other defining feature of Digital Image that you write about is the Kinetic feedback. The way I ready it, is that the matter interacts with itself to form of a feedback loop. I kind of understand that when thinking of computer software opening greater degree and range of aesthetic transformation, but what about a kinetic feedback when, let’s we say passively consuming content, by looking at a paining?
Thomas Nail
One of the interesting things in the book that I figured out by researching material structures is that some of the features of the digital images are common to the analog things, there not really this absolute division. If you think about digital culture as immaterial, in the cloud, virtual – it’s not. It’s fully material. A “cloud” is a huge building filled with hard drives. This vast Internet infrastructure all have material basis and in that sense it is still very analog. And in that sense analog still has many of these features as it has aspects of hybridity. An collage is an instance where you can break things up and reassemble it. You have a kind of hybridity in analog things. But as just in your example in looking at a painting there’s a feedback that happens, but we don’t often think of it as a feedback. We think of it as a noisy signal on a digital level, a negative feedback loop where we don’t want it to go. But that’s partially what interesting in analog and digital feedback is that it is taking us somewhere. There’s a feedback happening between two systems where both are sort of in control but neither are in total control, and the result is something genuinely unique (kind of simulacra experience, simulacra are meeting, refracting and making something new. When we think about looking at a painting we think of that as a passive reception of an external object. But the viewer is participating in that work of art just by being in that room, even if we’re talking at basic photodynamic level of photons radiating off your body as heat, and they are heating up at a very small level that painting. Light is reflecting off that painting and degrading it. By looking at a painting with light we’re destroying that paining at a very low level and over time it ends up totally destroying that painting and that’s why we have curation. Curators are in this unique position to really see and feel and understand the materiality of works of art. That’s a lot of what museum goes don’t think about. They feel like these are preserved work with ethereal structure to them. But the preservation process never ends, it’s ongoing. It has to constantly struggle agains the effects of decay, heat, and light-destruction of the painting. So I think they realize that the paining is more of a feedback loop that you think it is. And it’s also affecting you that you’re not fully aware either. Its light and coloration is making you more sensitive to subtle differences in light and coloration. Even if you think that you’re thinking about the symbolic meaning of such and such. A man by a river or something like that, or narcist looking at himself in the pond were thinking about symbolic representation of the paining yet there is a material basis that is also working on you that you might not be even thinking about, but it’s affecting you. And it is the same way with digital culture and the studies are now accumulating on that for sure. What is the Internet is doing to our brains? What is digital culture? How is it changing us? We’re using it for symbolic and representation purposes, but there is wast iceberg of material consequences to the environment, to our bodies, to our brain. To undergo the performance and the feedback that we enter into when we look at the screen and use some kind of digital device.
B.A. Gonczarek
When speaking of affecting and changing us by exposure to images, you see I’m in a business of supporting understanding, you can call it knowledge communication with the use of interactive whiteboards. And I have a front seat view on feedback loops and transformations of the content. I see how those work as a key to unlock human understanding. In the past the knowledge or concepts were conveyed by text paragraphs and static slides. Now those turn gradually into more visual forms, animation, ad-hoc drawing, into whatever works. So the way I see it, is that we’re on a path of getting away from the rigid, formal representations into a realm of smashing bits and pixels, so to say, to form new perspectives and gain new insights. I guess that’s in line of your thinking?
Thomas Nail
I think that’s right, I think that communication has significantly changed such that it is absolutely much more about feedback and with that feedback comes novelty. Feedback isn’t always what you want it to be. And with that what is interesting to me is that when images and words and material structures of how those are communicated – when you get all of those mixed together, when you have text, with digital speed of social media and users – when you get all of that together you’re getting some serious feedback transformation in which all of those are kind of pulled out of their original context and make possible new ideas that aren’t necessarily what we originally planned them to be. I think the feedback, even being explicitly interactive process, the interactivity makes us realize that we’re performing, that we’re doing something, not passively consuming. Even if we think that we’re passively consuming you’re actually generating something to. I think it makes us think deeply about the participatory nature that has always been the case with communication, visual or text-based that we’re involved in it, and that makes us responsible for intentionally shaping it, and not thinking that it’s this big structure and we cannot do anything. The mutability of communication is higher and more diverse that has ever been.
B.A. Gonczarek
Absolutely. The way I see feedback is that we always thought of the feedback on the cognitive level what worked? What triggered understanding? Was it a (so called) picture superiority effect where visuals work better then words, or spatial processing evolved in understanding of a concept or visual metaphors. But I guess thanks to your insights, I see that it’s possible to go deeper, beyond sensations to see the inner-working of three distinctive features of the digital image that you list: kinetic feedback, random motion and hybridity. So I wonder, from your perspective, do you see technology a one-directional enabler that gets us closer to the understanding of reality? Is it so?
Thomas Nail
That’s such a great question. On a one hand I want to say – it just depends on how you define digitality? But I think that the other definitions are typical ones of binary structure, so let me give you two answers to that question: Yes, digital world gets us closer to objective knowledge, more communication, transparency, more accuracy. Our pixels get so close now. The term ‘Retina’ it’s such a great term because that’s the limit where they eye can no longer distinguish the pixels. So what you could say on that front – yes, we’re definitely getting closer. Look how small the pixels are now, we are getting higher resolution and better accuracy on the world. If that’s a description – I disagree with that, i don’t think that’s why digitality is getting us any closer to reality or anything like that.
My answer would be – yes, I do think that it actually is but not in that way. I think that the thing that getting us closer to really thinking about reality in a different way, is that it’s forcing us to realize something that always been true about the nature of the image (whether analog or digital). The closer we get the closer we drill down to that binary structure of ones and zeros the more non-binary processes we start to discover. That’s what’s interesting about digital. It’s the actual conclusion that if we push it far enough we see it break down and see that below that it’s actual continuous fluctuation of quantum processes that are not under control. And this reveals to us something novel about matter itself. Something that always been novel, but we haven’t comforted it in that precise way. The history of art and media is typically Humans trying to control the world and make it look their way, and do it certain something. There is a minor history to be said there, but for the most part the western history of media and use the technology is to control the nature. But what’s interesting to me is that we’ve reached limits of that control and we’re forced to realize that it is impossible project and what we’re really have been doing is not successful domination to completely get access to objective reality but that we’ve been engaged in this kind of feedback look where materiality of media has shaped our bodies, our senses, our brains just as we’ve been shaping the world thought all of this media – and that’s what I think the truth is to be realize in the digital age. It’s not the superiority of the digital image but precisely what the digital image is exposed to us explicitly. So we have to confront that fact.
B.A. Gonczarek
It’s certainly getting our thinking less infantile, but do you see any risks that we might not be aware on this path?
Thomas Nail
For sure, the risk is that we will keep trying to find the ultimate way to bypass material processes and the performative act of interpretation. What I mean there is that if you think you can break down the world into totally discreet bits and bytes – that’s the danger, because it will drive you absolutely mad trying to produce a clean-cut distinctions between ones and zeros and not realize that there is this material process that will always spoil this effort. The danger is to use technology and media to try to control and essentially dominate meaning and leave out interpretation. Some philosophers that really herald the digital age they imagine – oh, we’ll just put jacks into our heads and we will just communicate with binary code, and that we’ll bypass all of the messiness of the language. I said this word that might mean something different to me than it means to you and we have base for this messiness which is, in truth, the beauty of poetry and literature. We can just get rid of all of that and just have purely objective truth with binary code. And I think that is the danger, thinking that you can avoid the material and what we call ‘an interpretation’ but it’s essentially performative, collective feedback that is generating something news not understanding some objective state of the world
B.A. Gonczarek
Before we close, what you see as a possible outcome of increasing software capacity in transforming digital images? Given the nature of digital image, what do you expect to happen in the near future?
Thomas Nail
Guess this depends how pessimistic or optimistic I am.
B.A. Gonczarek
Give us your best shot.
Thomas Nail
I’ll give you both. What I expect when I’m feeling pessimistic is that we will continue with quantum computing to try practically to keep pursuing to break world down into ones and zeros and master quantum flaws and erase any errors, any noise, any fluctuations which we don not want to happen in electron flow. That we will keep on that path and try to continually break things down in an attempt for absolutely non-interpretive objective reality. To think that we’re getting closer to that is to me absolutely the danger when I feel pessimistic.
Optimistically I think that technologies that emphasize and take seriously the materiality of the media that they using, (not just as a neutral media to facilitate communication, but as itself a creative thing, something that is changing the world). To recognize the changes that it’s having both on the environment, on material world, but also changes that it’s producing in us, in our bodies – and to take that seriously and ethically to treat it more as a work of art. The sharing of images, the sharing of text is not neutral communication, it’s transformative, it’s doing something to us. I think that if you think that’s its neutral communication that’s subjective you’re going to miss that ethical moment. So you’re really need to think about that ethical moment. We’re responsible for what we’re doing to ourselves and what we’re doing to each other, and what we’re creating. So taking ownership of that essentially and we supposed to be being serious and intentional of what that is the optimistic outcome
B.A. Gonczarek
And I join you on this optimistic end. Listen Thomas, it was a great pleasure to talk to you today. It certainly helped me in my exploration of verbal-visual field of communication. But I also believe that your perspective is fresh to anyone that is trying to understand the direction that technology is taking. Many thanks for sharing your insights with us today! And good luck shaking off human’s immaturity of perception!
Thomas Nail
Thank you.
Crafting the right tools for creatives in the digital age. A conversation with Adam Wiggins, entrepreneur and the director of a research lab
A conversation with the director of a computer research center about the future of digitally-supported thinking. On developing ideas and why digital tools are not yet delivering in that realm. What the workshop of Thomas Edison or the works of Albrecht Durer teach us about the environment of thought? Why does it seem that we’re missing a software category for open-ended thinking and is it possible to demand from tech that it remain more human?
B.A. Gonczarek
I’m here with Adam Wiggins. Thrilled to discuss the future of pen and paper in the digital world. Adam is a perfect person to learn from, and he’s a director of a research lab that works on digital tools of the future. Welcome, Adam.
Adam Wiggins
Thanks for having me.
B.A. Gonczarek
For our listeners to understand from where you’re coming from – among other interests Adam co-founded Heroku, a platform for deploying and running apps, acquired by Salesforce in 2010, at some point after successful exit Adam relocated from San Francisco to Berlin. I must admit I read your essay about that decision and I‘m taken by the comparisons you describe between US and European startup communities. Adam, outside of his advising opportunities here in Europe, for over four years a part of research lab Ink&Switch. That’s where we met as we’re both focused on innovative computer-mediated forms of teaching and working. Your vantage point, Adam, gives as great opportunity to discuss – what lies ahead in terms of interfaces and computer-supported creativity. Before we get there though, describe, if you will, your research lab focus and expand the story that I drafted on how you got where you are today.
Adam Wiggins
Yeah, happy to. Ink&Switch is an independent research lab. And our theme or our area of focus is kind of what computer use will look like in 10 years, and particular creative or productive computer use. So by that, I mean, word processing, spreadsheets, Photoshop, or kind of the traditional kinds of things you would think of, or maybe more broadly thinking, creating productive tools. And that kind of standing in contrast to the, what I call consumptive uses of computers that have taken over a lot more of kind of, let’s say, this tech world’s focus in recent years, Facebook, Netflix, YouTube, social media, etc. And so we’re kind of chartered with thinking about, you know, what does that what does that future look like? And how can we, how can we make it better for creators?
Yeah, and then in terms of how I got here, you know, I’m a serial entrepreneur. But I guess you’d say the theme of my career has always been that I love making tools that help other people make things just get a particular thrill out of out of guests, done everything from payment gateways to fabric, retail, and then developer tools. Probably the most successful of all of these was a cloud deployment company called Heroku. And there that was basically just a tool for taking a web app that you’ve written and getting it online with the push of a button. And that was quite successful. And actually, that gave me a lot of the both the sort of financial freedom but also the, the ability to kind of leave the Bay Area and go work on something a little, little weird, a little frengie, you know, not as sort of not as connected to Silicon Valley doesn’t pay as well. But for me, at least, had some more intrinsic interest and potential more benefit to humanity over the long term.
B.A. Gonczarek
And why Europe was the choice?
Adam Wiggins
Yeah, it was a little bit of a process of elimination, which is I wanted to go somewhere place interesting after having spent most of my life in California, and seven years in San Francisco, and really ready for change of pace, I really wanted to be creatively refreshed, much as I love the Bay Area, and a lot of what was emerging in the startup world at that time. I also felt like it’s hard to have new original ideas because there’s a kind of group think that naturally emerges when you get a bunch of the same kind of people together, and in the same place, working on the same kinds of problems. But I still wanted to do something connected to the startup world, there’s really not that many cities that are great for that. It’s kind of like San Francisco, New York, London and Berlin, then had kind of an up and coming scene. And of those four, Berlin seemed the most interesting. So I went there to do some short term gigs and advising and kind of fell in love with the place and still here five years later.
B.A. Gonczarek
You describe yourself as a digital toolmaker, and I understand your drive toward creative tools for those that want to be creatives. That’s also the focus of your research lab. What’s your routine for developing ideas?
Adam Wiggins
Yeah, well, part of the way I’ve thought a lot about sort of how to help people develop ideas and introspect it on my own process, as well as that’s, that’s an area of study for our lab, we’ve done a lot of user studies, user research around, what does the creative process actually look like? So for myself, you know, I’ve realized in this I’m definitely a what I would call a deep work type. So I kind of identify a problem or an opportunity that needs deep thinking, usually, that comes in the form of suspicion or some weird knob in the brain kind of feels like a pebble in the shoe, and feels like it needs attention. And I’ll just set aside, usually in the morning, which is when I’m kind of most fresh and creative, anywhere from 30 minutes to several hours, or longer, you know, however long it takes to tackle the problem. And there’s a process that involves lots of raw material, everything from web research and sketches, notes I’ve taken on my phone, and then bringing all that together in a place, I can kind of see it all together. And then using that to kind of riff on the idea, extend it, ask questions. And that brings us to something we’ve been spending a lot of time thinking about, which is what we usually call externalize thinkers, which probably has some good overlap with visual thinkers. But it’s just basically if you’re the kind of person that tends to use a notebook, to kind of write down your ideas or sketch out your ideas, or for myself, for example, when I’m in a meeting, I feel more productive when there’s a whiteboard there, where you can kind of extend the conversation into this visual space, write things down scribble, and so on to supplement the group discussion so that folks who are drawn to that which is not everyone, but there’s a good chunk of creative people, we usually call externalize thinkers, and I’m, I’m definitely one of those I do these kind of deep work sessions where I use externalize thinking through pen and paper or my iPad to my computer, and, and so on, kind of make a big mess with lots of inputs, use that find patterns, get insights and ultimately hopefully make a decision or develop my idea in a way that allows me to move forward with it.
B.A. Gonczarek
Right, with what kind of software you’re currently supporting your process of ideation?
Adam Wiggins
Yeah, so I have used all kinds of things over the years everything from kind of Dropbox, like my main kind of archive, long term, personal corpus, many, many types of note taking apps, more recently with the tablet and stylist for impact factors, sketching stuff. Currently, one of our research tracks that we’ve pursued in the lab of the last few years is sort of this thing where you have a freeform canvas where you can move these media cards around kind of mix different media types and sketch at the same time. And actually, at the moment, we’re trying to take that line of research, which we’ve written a few long manuscripts about, and actually spin that out into a commercial product. So with any luck in the in the coming months, the project we’re currently trying to commercialize, which is called Muse might be available in the app store for the iPad. So I tend to use that a lot, because we’re trying to test out the viability of that as a real world product.
B.A. Gonczarek
I wonder what kind of category of software you think of when you think of Muse. From my observation this creative mode of thinking is very little supported by existing software categories. What are your thoughts on that?
Adam Wiggins
Yeah, that’s, that’s a remarkable thing. I’ve noted ideation, sometimes you call that that early stage, right? When you think of most digital tools, like a word processor, or a photo editor, or a diagram tool, we call those authoring tools. So you kind of already know what you need to say, and now you want to get it written down, or get it into a presentation or get it into some shareable format. But the part where you need to actually figure it out in the first place. It’s a place it’s never been, I think, super well supported by digital tools. And maybe it’s just that, you know, computers are fundamentally kind of structured and rigid. They’re good for databases and tables and things. But the the creative process, at least mine is just very messy and freeform and all over the place. And this is where analog tools, the pen and paper, the whiteboard, the printouts that you can kind of scattered around and reorder the pages post it’s on a wall. That stuff I think better maps to the the creative process. And I don’t think it’s infeasible that computers can do that. And in fact, that’s, you know, part of what we’re trying to pursue with muse. But I think not many tools have tried to do that, right? You either have very kind of visual and freeform, relatively visual and freeform things like a illustrator or Figma or something like that, they tend to be very focused on the visual side. Whereas I think realistically, most of your thinking, deep work kind of stuff tends to be lots and lots of text. Of course, you can do that in a text editor, a word processor, something like your note taking app. But then you also need the sketching, sketching and scribbling to somehow bringing both of those things together, I think it’s not something that’s ever been a great fit for digital tools. But it’s also part of the reason why at the lab, we’re excited about tablet form factors, we spent a lot of time evaluating the different platforms there, including iPad and Chrome OS and the Microsoft Surface and so forth. Because I think the different form factor and the different posture that you use a tablet and stylist maybe but it’s kind of our working hypothesis is that may be more suitable to that kind of freeform ideation. But fundamentally, computers tend to become from this more like rigid and structured kind of approach.
B.A. Gonczarek
I fully agree with you there, because the tools that we use today are descendants of those computer technologies that were created to supporting processes where you don’t want to overstep boundaries, you just want to get the thing flowing nicely. For each input there’s a certain output. What you’re describing here, however, is kind of this divergent thinking mode of operating where you explore possibilities and probably later arrive at conclusion that can be processed using those old fashion, rigid systems. You know, when we speak with people that make use of whiteboards they tell us that, for example in sales, they tell us that the single best thing that can happen during your sales conversation is having your customer reaching for a piece of paper or a whiteboard and star drawing with you, exchanging thoughts, because that illustrates engagement. And current multitouch devices, are good enough to support that but yet they are not seen as devices to support divergent thinking, rather they are seen as devices for content consumption. Do you see it in the similar way?
Adam Wiggins
Yeah, that’s a great point. I think, generally speaking in a conversation or sales meeting, someone going to their device is not seen as a good sign that things are going well. But then walking up to the whiteboard does indicate a kind of an opening up a, we’re going to think through a problem together, we’re going to think through something together, which is an interesting connection. And then yeah, the divergent versus convergent thing is is also like that phrasing or labeling the idea of divergent thinking being you’re kind of just everything’s open, you’re trying to widen up. Sometimes I talk about like Double Diamond. It’s kind of like that opening and then closing but some point you need to narrow down and select and so on. And you could say that perhaps the kind of the digital tools we have today tend towards being better for convergent thinking. And what we lack is the divergent thinking that’s not no one’s really quite cracked yet.
B.A. Gonczarek
Exactly. And also I think that we’re currently in a situation where creatives when they want to go visual are forced to use tools dedicated to processional designers. But in real live every message, every pitch is an invitation for expression, don’t you think?
Adam Wiggins
Yeah, absolutely. And I’ve even struggled with using the word creative creator creative professional, like finding the right terminology, because I think it is easy when you hear creative to think I don’t know about being a designer that’s doing paint in some kind of mysterious process. But I think of it like all of those wider array of knowledge worker creative professional things, including budgeting, sales process, making a presentation, software engineering, doing database schema or something these these are in the broader sense of the word creative activities in the sense that they create. And they do require, they do depend on ideas, and they do depend on divergent thinking, the best people at their jobs are the kind of people who can make those unusual leaps and have unusual insights. And it’s it’s not a it’s not a straight, it’s not a straight line from the problem to the best solution. Maybe.
B.A. Gonczarek
Absolutely, as everything starts with an idea in the end. When I hear what pioneers of visual communication, let say David Sibbet, have to say, he mentions often that visuals in business act as props on the stage on which management sells their visions. Also, the expression “I see” stands for understanding, so to me it demands visual expression.
Adam Wiggins
Yeah, I mean, I, you know, the many of the most powerful ideas I’ve encountered in philosophy and life, as well as in business do have some kind of visual, visual expression I think of like Maslow’s hierarchy of needs, is one example. There’s plenty examples of those in the business world. I don’t know Steve Jobs famously like boiling down the cluttered Apple product line when he came back to the sort of the two dimensional grid and essentially gives you four products. And that kind of very simple breakdown can be just be incredibly powerful. Of course, you can also lampoon that a little bit I think the HBO Silicon Valley show had this goofy CEO guy that had his convergent triangles of whatever that was just like a nonsense visual expression, sort of visual diagram sort of making fun of that trend. But but it’s sort of something that people are so aware of, because a truly great visualization can really help you more often can really help both express and help you grasp an idea that that words alone, for example, wouldn’t be able to.
B.A. Gonczarek
And I believe we are all well rooted for that. If you pull a quote from Aristotle – “without an image thinking is impossible”, that kind of explains it right? But I wonder what you think about what creatives and their environment could tell us about a digital environment for open-ended thinking? I saw a great article on your research lab page about physical workspaces that are stimulating for creatives, supporting tendencies for nesting things, availability of space and using messy free forms for expression that you mention that you use as well.
Adam Wiggins
That’s right. Yeah, we took a ton of inspiration from physical world sort of crafting spaces, work spaces, fluid free form a little informal, little personal little messy, which, which all of those things don’t tend to be what we get in our digital tools. Personally, you know, I love reading what I usually call maker biographies of people who create things that kind of the process of, you know, that heartbreak and challenge and struggle of how to make something, write a book, do science, invent something and entrepreneur, that sort of thing. And historic ones I find especially fun, because often their tools are maybe quaint by our standards, but they’re often quite interesting and beautiful, famous examples is something like Leonardo da Vinci with his notebooks or meet Marie Curie, his notebooks are still, you know, still reading radioactive to this day. And in that category for especially for stuff, it’s new enough, but we have photographs, you go look at, for example, the Thomas Edison’s invention workshop, or various sort of architects or artists studios. Now he’s got the feeling that kind of like a nest, their personal, they have some selection of tools, that’s exactly what they need for their craft. You know, if it’s an architect, maybe that’s drafting tables and straight edges with its artists, maybe it’s brushes, it’s someone more kind of word oriented, maybe it’s books, they’re often a little cluttered, they’re often little messy, they seem like it’s but place you can try things, you can experiment, you can you can mess around, basically, while you’re exploring an idea. And again, none of that it maybe you get a little hint of that. And so how some people treat their, you know, on a desktop operating system, like a Mac, Windows, Linux, the way that people kind of scatter files all over their desktop screen could be a little bit of that feeling. But I love looking at those kind of those. Yeah, there’s old school studios or what have you. Something about that is not necessarily being messy, but something about the nesting the personal the selection of tools and the space to experiment. I think that’s something we certainly been taking experiment, inspiration from, with Muse and our various previous incarnations of the ideation tools.
B.A. Gonczarek
I’m sure you’ll agree that current tools don’t invite messiness to fuel that type of thinking.
Adam Wiggins
Yeah, what would you can see as a feature, I do see as a feature that, you know, they provide structure and, and, and uniformity and normalization, I think that’s great for certain things. You know, I don’t think I’d want a messy Trello board the point of Trello, is that it helps you get a little, you know, organized and structured. But if you’re, again, if you’re talking about that ideation, divergent phase, that’s where maybe that’s an impediment rather than a benefit.
B.A. Gonczarek
Absolutely, structre is great but later in the process, after you arrive at an idea. And you mentioned Leonardo, he famously suggested to look for inspiration in a splash of paint to start from. So when thinking about free flow of ideas, the way we work, is that we put things on a canvas of a whiteboard to draw connections similarly as you do with Muse so I see an ocean of similarities here. And also, when thinking about inspirations coming from pre-digital age I often think of Albrecht Durer. He encouraged students to express ideas visually using strokes, not in text, but with strokes. I wonder where those findings get us when thinking of computer interfaces of the future?
Adam Wiggins
Yeah, well, obviously, it’s, since I’ve said it’s part of our labs job to think about that something I’ve put a lot of thought into. You know, clearly mobile and touch screens have set a new standard, a new expectation in terms of always on internet, a great lot of battery life and screens you can interact with, and, and just an integration to the world, right that we take our phones with us and we pop them open for for a photo or a note or something like that. And that kind of trend continues with the voice assistance things. And you know, we’ll see how that actually works out. But we definitely see this inner leaving of digital and physical worlds. But my question there in thinking about the future of UI, okay, that’s all great for again, these common sort of more consumer uses, like sending message calling yourself a rideshare social media playing podcast. But when it comes to creators, it’s still all about the keyboard and mouse, or a keyboard and some kind of pointing device, Wacom tablet or something. And someone good with these on a desktop computer is so fast, so precise, so capable, touch screens, and turn mobile interfaces are just slow and comes clumsy by comparison. But then I go will keyboard a mouse still what will be using, say in the decade? And I think that’s questionable, because if nothing else, you’ve got a generation of people who grew up on touchscreens are not really comfortable with with keyboard and mouse.
B.A. Gonczarek
Do you think that we’re still in situation that we’re confined to the interfaces we started with? We began with character-based consoles of the computer terminals, as they were called, but gradually restrictions for other types of input were removed yet we’re still seem to stuck with paradigms of legacy UI.
Adam Wiggins
Yeah, well, I wouldn’t necessarily say stuck with it’s that they’re, they’re really, really good, right, they stood the test of time, and people are powerful with them. And software has been built, assuming them and that sort of thing. And I think we’re still collectively as an industry trying to grasp how we can use these new capabilities, or the new expectations that you younger generations will have. And I think, you know, there’s interesting experiments with, you know, I don’t know, Google jam board and Microsoft Surface Hub with sort of big touch screens that go on the wall that you can touch with several people together, and certainly the computer interaction community that are allowed a little bit of part of his has done a lot on that, but we kind of haven’t quite cracked it, we haven’t really found the thing that is truly going to unlock some new level of productivity that you don’t have a desktop computer with keyboard and mouse.
I think the thing I would be excited about just kind of speculatively is is something that does draw from those Maker Studios, we talked about before those NES those workspaces, where it’s something that uses some of this kind of touchscreen, maybe even voice control capability, but it’s more of a room sized workspace. And something where you’ve got maybe multiple screens, which includes small ones you carry, or bigger ones that are, you know, a drafting table form factors, something on the wall, certainly being willing to embrace multi touch styles, to use two hands of once, right, like, that’s something we’ve written about a lot, which is, you know, phone apps tend to be designed for that one handed use, just do it with your thumb. But in this kind of work space, you know, we got 10 fingers, let’s use them all, right? And that, that, that if we cut free a little bit, both try to grasp the new kind of capabilities that we have from the hardware and the operating systems, but also kind of predicate free of that legacy of a small screen on go. Just do it with one hand, you bring in the fingers, the voice the stylus. You know, maybe in some Hollywood version of this is like you know, Tony Stark and his Iron Man blab talking to Jarvis and touching, touching screen. You know, that’s all obviously very Hollywood, but I think I see some version of that. And if you tie it in with the remote work trend a little bit what we have with video chat, group chat, share documents, real time collaboration, you put that together with some kind of room size workspace that uses these new interfaces, but also cuts free of that the old mobile assumptions and gives you more powerful gesture libraries, that starts to become a picture the least that I can get excited about for where where computers for creation might be in 10 years.
B.A. Gonczarek
Do you think we would not be intimidated by those capabilities? Using gestures and visual techniques? The way I see this is that kindergarten kids in schools they can explain everything with a simple drawing. But the process of communicating visually is kind of unlearned during later stages of education in which use of words is more emphasizes than use of images.
Adam Wiggins
Yeah, that’s interesting. We definitely when you look at the academic tradition, which includes everything from you know, how you write a master’s thesis or you know, papers you consume in the scientific world, and all the homework we do kind of in the earlier stages of kind of school life. Yeah, they tend to be very, very word oriented and words and symbolic representation more generally, that includes mathematics, for example, is incredibly powerful, and really good. But yeah, I probably agree with you that that maybe there’s a an overemphasis on that, and it depends on the idea being expressed. And it depends on the person that’s expressing it or trying to express it, too. But yeah, certainly trying to maybe recapture some of that childhood reach for the natural visual expression is something I can really get behind.
B.A. Gonczarek
On the one hand we have limitations of current UI that kind of draw from the past and we kind of start to allow for free flow of thinking using interactive whiteboards or solutions like the one you’re working on currently. On the other hand we have users of those solutions that might not be fully ready to appreciate those solutions when they arrive because of visual skills being left behind in later phases of person’s education. This reminds me a situation back again with Durer, the painter and the theorist of the German Renessaince. He was known for working around constrains of the technology of his time. The printing press was kind of brand new during his days. So he worked to add graphical doodles and free hand movement to the more rigid, printed materials. Those are now understood by scholars as expression of his creativity and desire for free expression of his ideas. Don’t you think we’re both in similar situation with digital technologies of our times?
Adam Wiggins
That’s an interesting comparison, obviously, the printing press and type set, you know, these these literal rigid rows that you’d sort of plug pre pre stamped out, made of metal into is the definition of being very structured and takes you away from that freeform doodling and notably what came before. I love these texts that were kind of hand made from the 13th 14th century, where you have the handwritten text, but then there’s always the what they call the illumination, which is the essentially they’re just doodles in the margin, but they’re, they’re gorgeously beautiful, they illustrate sometimes, there’s something to do with the text, or they just illustrate something about the time and again, something like we lose some of that free foreignness and some of that visualness when we do go to the structured the structure to the printing press, and obviously with the benefits outweighs the losses that we could mass mass produce this stuff. But yeah, it’d be great to see we can find find the technologies to bring that back a little bit.
B.A. Gonczarek
I’m surely on your side hoping for that! Fast forward 20 years, let me know what you think of the division of labor between humans and machines. As we see steady progress in AI technologies and I wonder what do you think, when thinking about computer technologies, how progress with AI affects our choices when preparing for the future.
Adam Wiggins
Yeah, well, you can call me probably a little bit more skeptical of the the impact of AI in the maybe in the near term, at the very least. Although I think it’s definitely interesting to think about, I mean, you know, one observation is that machines are often better at what we might call brain work than physical work. So like, I don’t know, auto correction or detecting credit card fraud, or even something like self checkout in the grocery store those those work a lot better than actually sort of grasping and manipulating the physical object. And I certainly hope that we’re I think it’s, it will be good in the long term, even though it hurts some folks jobless in the short term that more and more machines will do those, let’s call them wrote kind of thinking jobs.
But yeah, when it comes to creative activities, I’m very skeptical machines matching humans anytime soon, maybe ever, and probably reflects at least a somewhat irrational faith in the uniqueness of the human spirits are just something I want to believe rather than something I have a good rational basis for. But I do feel like computers do well with well defined inputs and outputs and repeating patterns of the past. But creativity is fundamentally about inspiration about walking off the beaten path about surprising insights, doesn’t have clean poets, it doesn’t have clean outputs. And maybe also, it’s just that an inspired creation is something that wakes something in another human. And for the foreseeable future, it’s hard to imagine that will be done by something other than other humans. But of course, machines have a huge role. There’s the tools, the digital equivalent of pen and paper, they’ll also be in a way, I think, collaborators for us just like a grammar checker in or an auto summarization algorithm helps out and author. And you see that the design world, the architectural world with kind of algorithmic design, it doesn’t seem to reduce the role of the human, but it does change it maybe a little bit. And you can talk about having like a collaboration between the unique capabilities of computers, particularly as they get able to do more higher level thinking tasks, we might call it and the human which can guide them to the things and guide the computer to the things that matter for humans and bring that kind of weird spire spark, that that fundamental, different creativity thing that is, again, I think, unique to humans.
B.A. Gonczarek
Supporting creativity makes as able to stay human in the future. Even what we think of as creative outputs might be considered a glitch in logic from a perspective of AI, but it’s on the other hand a valuable asset that we have. Don’t you feel it that way?
Adam Wiggins
Yeah, well, again, I feel it, I don’t know if that’s, you know, intellectually defensible. But, yeah, well, yeah, for sure. I mean, maybe it’s partially at least that just I love both the creative process for myself, but also, again, you know, I got a whole lab that’s huge track research is dedicated to understanding and intersecting that seeing it at work in people. And it does really feel like there’s something special there to me not say it’s something that can’t be replicated by machines. But I think it would just be a different thing if it is done by a machine because I work in fundamentally different ways. And therefore, you know, that’s a reason to be positive about a future where there is a role for what machines can bring to the table like generative or algorithmic design, and a role for the unique things humans do, perhaps with less of the road work and then maybe together we can create something better than either of us could individually.
B.A. Gonczarek
Absolutely, although I still fear we might arrive in entirely deterministic world where no splash of paint is possible to open new possibilities. Adam thank you for being with us today and sharing your perspective.
Adam Wiggins
Really enjoyed it. Thanks so much.
