The nature of digital image
A conversation between Thomas Nail, an Associate Professor of Philosophy at the University of Denver and the author of the recently published book “Theory of the Image,” and B.A. Gonczarek, your host.
A philosopher’s perspective on the nature of digital images, their material roots, and various consequences which escape our consciousness. Why the digital is more analog and material than we think and how the origins of this revelation go back to Rome. How viewing a painting makes us a part of it? An attempt to explain communication on a more fundamental level than the cognitive. How we’re progressing with the development of technology, how new frameworks can support our understanding, and how we continue to risk missing the point with existing frameworks.
Listen to the conversation:
Transcript
B.A. Gonczarek
I’m here with Thomas Nail, the Associate Professor of Philosophy at the University of Denver and author of recently published book Theory of the Image, welcome Thomas. I must admit I was really looking forward to our discussion. When preparing to our conversation I did my research online and I was taken by how well you’re received by your students. You students describe you as very knowledgeable and approachable. Your openness is something I experienced myself, so thank you for the opportunity of doing this podcast together. And to explain to our listeners – what we’re trying to do here is to (possibly) bridge the gap between abstract thinking and acting, between thought and execution by an exchange between you, as a philosopher and me, as digital toolmaker on a topic of digital image.
My main goal for today is to hear your point of view on the future and possibilities that technology gradually unlocks. Now, I’m aware that the digital image is only a short chapter of your recent publication but I believe that limited scope of our discussion is enough to inspire our listeners. After all, we’re all users of digital devices don’t we.
To begin, describe to us, if you will, your way of working. What is New Realism and what is your method of approaching problems?
Thomas Nail
If I had to sum up main findings of the book that guides the whole project is that the image we often think about as a mental representation, something as in our brain (in our minds) which is a copy or resemblance of the world outside. I think that’s not right, there’s definitely something going on but that’s a very narrow way of thinking about what an image is.
An image is a real thing, it is something that happens in our eyes and in our brains, that is related to the external world, but that is a tip of an enormous iceberg. That’s the part that we see on the surface. Below the surface of the water is this enormous process of the rest of the world, of the enormous processes that we don’t actually see which are part of the fabric of the world and forms and media that we use, and it’s very active. What we have in our brains is not a copy of the world, it is the world itself just by other means. It is a continuation of the world inside of us just. It’s not a question of resemblance but interactivity, of performativity. We are interacting with the world when we see, although we often experience vision as a passive thing that sort of happen to us, but that’s actually very active both in our bodies (in our eyes the way they seek out, move and follow and respond to the world). One of the main takeaways was to think about much larger context what an image is but also what the world does. Whether are humans there, or not, there are images, as they sort of they engage each other. The way we interact with the world those images interact with themselves and that interaction is what produces an image. That’s a broad definition of an image but the shift is from thinking about images as representation to thinking about images as processes with their own habits, cycles, they sort of interate and respond to each other to produce meta-stable states. They are flowing and moving, but they are also stabilized, so they look static. If you look at an object on a table – it looks like it is just sitting there but it’s not. And even when we weren’t looking at it, the image is still because the image is real and material whenever we think about that. We’re part of it when we view it.
B.A. Gonczarek
As you describe in the book the image is a process by which matter twists, folds, bends and reflects itself into sensations and affections. What was the inspiration to arrive at such viewpoint on the image itself?
Thomas Nail
It’s an old inspiration actually, it goes back to the Rome and poet Lucretious. We only have one book of his philosophical poetry – De Rerum Natura. In the book, inspired by Epicurus, who said that the earliest Theory of the image as a material process that we have in the history of the west and it’s since been transformed by other ideas, but I do think there’s something to go back to. For me the inspiration was his poetry and ‘Simulacra’ – Everything in the world is radiating out images. Images are bouncing off each other, eventually they get to the exterior and fly off to collide mid air with other images. Some people interpret it as there’s ghosts flying off of things, but that’s not what he says at all, it’s actually closer to modern physics and light. He didn’t use the language of photons, he used language of simulacra but that’s essentially what it is – that things inside of themselves are vibrating with photons. Photons are heat, photons are light, they are constantly vibrating and release waves of photons, and photons collide in mid-air. And for that reason at every stage they actually are making something, they performing and producing. There’s no resemblance, but no genuine copies, no originals, there’s just these singular processes that refract (like you drop two paddles into a pond and the ripples would key each other and make a new pattern – at every stage you’re always looking at some specific pattern of the photons interact with each other. So it’s a very materialist way of thinking about what an image is as opposed to the idealist way, which is – it’s an idea I have in my brain. And if that’s what you think an image is then only humans have them, only humans can sort of talk about them and they always will fail in representing the original image. There will always be some poorly construed copy of what’s out there. If you think of an image of a real, material, singular process, then it changes the way you think about what an image is. What an image is is what an image does. It doesn’t represent anything, it moves, it does. So the question is – what are the patterns? That’s why I think the visual aspect makes a lot of sense because to understand what images are you need to have an interactive and visual tools to map out what that image is doing.
B.A. Gonczarek
And I believe that’s also applicable to the digital world. I found it actually fascinating of how you’re shifting perspective here. You name three features of mobile nature of the image and I’d like to ask you about hybridity that you list as one of those defining features. You call it a pinnacle of fragmentation, I’m curious what opportunities fragmentation opens`?
Thomas Nail
When I say fragmentation I don’t necessarily mean complete isolated fragments. They are little knots and pieces of strings, always related and connected with another pieces so that the pieces are never fully cut off from one another. This is the way people tend to think about digitally as just fragmented bits and bites, ones and zeros – but there are no fragmented ones and zeros that are fully cut off. That sounds opposite to the definition of what we think of binary. The truth is if you just dig below that level – is a signal on or of (basis of digital communication) and look at the material structure of transistor – it doesn’t work like that. There is a constant flow of electrons and photons moving through that transistor and they do not always stop at the gate when the signal is supposed to be off – they jump the gate. It’s a quantum effect called tunneling in which electron movement actually passes the barrier. The smaller technology gets, the more data we can store, the tinier the gates get. And the tinier the gates get the easier it is for the flor of electrons to pass through the gate and then you get an error, and your computer crashes. And these are happening more often than used to because of the technology. When your computer crashes there’s a good chance that’s because of the quantum effects of the material movement of the electrons. So thinking about all these pieces it really draws your attention to the creativity and the agency of the matter itself that we’re dealing with. We try to represent things of ones and zeros but what we’re often encountering is this very fascinating resistance of the matter itself and that opens new possibilities of working with that matter as opposed to trying to dominate it and trying to stick it into a binary code. Oneinteresting question for the digital age and XXI century is what new things might we discover? What new visual or communication aspects if we let the materiality, if you will, to play a role and speak instead of trying to silence it or make it your bidding. What might it say to us? How might we use it by working with it as supposed to trying to master it.
B.A. Gonczarek
Absolutely. I remember from your book when speaking of hybridity you touch the digital foundation of the image saying that
“Anything that can be coded can be transcoded and then turned into a hybrid of something else.” So the beauty of transformation and allowing for new thing to arise from something that preexisted before opens a lot of new possibilities.
The other defining feature of Digital Image that you write about is the Kinetic feedback. The way I ready it, is that the matter interacts with itself to form of a feedback loop. I kind of understand that when thinking of computer software opening greater degree and range of aesthetic transformation, but what about a kinetic feedback when, let’s we say passively consuming content, by looking at a paining?
Thomas Nail
One of the interesting things in the book that I figured out by researching material structures is that some of the features of the digital images are common to the analog things, there not really this absolute division. If you think about digital culture as immaterial, in the cloud, virtual – it’s not. It’s fully material. A “cloud” is a huge building filled with hard drives. This vast Internet infrastructure all have material basis and in that sense it is still very analog. And in that sense analog still has many of these features as it has aspects of hybridity. An collage is an instance where you can break things up and reassemble it. You have a kind of hybridity in analog things. But as just in your example in looking at a painting there’s a feedback that happens, but we don’t often think of it as a feedback. We think of it as a noisy signal on a digital level, a negative feedback loop where we don’t want it to go. But that’s partially what interesting in analog and digital feedback is that it is taking us somewhere. There’s a feedback happening between two systems where both are sort of in control but neither are in total control, and the result is something genuinely unique (kind of simulacra experience, simulacra are meeting, refracting and making something new. When we think about looking at a painting we think of that as a passive reception of an external object. But the viewer is participating in that work of art just by being in that room, even if we’re talking at basic photodynamic level of photons radiating off your body as heat, and they are heating up at a very small level that painting. Light is reflecting off that painting and degrading it. By looking at a painting with light we’re destroying that paining at a very low level and over time it ends up totally destroying that painting and that’s why we have curation. Curators are in this unique position to really see and feel and understand the materiality of works of art. That’s a lot of what museum goes don’t think about. They feel like these are preserved work with ethereal structure to them. But the preservation process never ends, it’s ongoing. It has to constantly struggle agains the effects of decay, heat, and light-destruction of the painting. So I think they realize that the paining is more of a feedback loop that you think it is. And it’s also affecting you that you’re not fully aware either. Its light and coloration is making you more sensitive to subtle differences in light and coloration. Even if you think that you’re thinking about the symbolic meaning of such and such. A man by a river or something like that, or narcist looking at himself in the pond were thinking about symbolic representation of the paining yet there is a material basis that is also working on you that you might not be even thinking about, but it’s affecting you. And it is the same way with digital culture and the studies are now accumulating on that for sure. What is the Internet is doing to our brains? What is digital culture? How is it changing us? We’re using it for symbolic and representation purposes, but there is wast iceberg of material consequences to the environment, to our bodies, to our brain. To undergo the performance and the feedback that we enter into when we look at the screen and use some kind of digital device.
B.A. Gonczarek
When speaking of affecting and changing us by exposure to images, you see I’m in a business of supporting understanding, you can call it knowledge communication with the use of interactive whiteboards. And I have a front seat view on feedback loops and transformations of the content. I see how those work as a key to unlock human understanding. In the past the knowledge or concepts were conveyed by text paragraphs and static slides. Now those turn gradually into more visual forms, animation, ad-hoc drawing, into whatever works. So the way I see it, is that we’re on a path of getting away from the rigid, formal representations into a realm of smashing bits and pixels, so to say, to form new perspectives and gain new insights. I guess that’s in line of your thinking?
Thomas Nail
I think that’s right, I think that communication has significantly changed such that it is absolutely much more about feedback and with that feedback comes novelty. Feedback isn’t always what you want it to be. And with that what is interesting to me is that when images and words and material structures of how those are communicated – when you get all of those mixed together, when you have text, with digital speed of social media and users – when you get all of that together you’re getting some serious feedback transformation in which all of those are kind of pulled out of their original context and make possible new ideas that aren’t necessarily what we originally planned them to be. I think the feedback, even being explicitly interactive process, the interactivity makes us realize that we’re performing, that we’re doing something, not passively consuming. Even if we think that we’re passively consuming you’re actually generating something to. I think it makes us think deeply about the participatory nature that has always been the case with communication, visual or text-based that we’re involved in it, and that makes us responsible for intentionally shaping it, and not thinking that it’s this big structure and we cannot do anything. The mutability of communication is higher and more diverse that has ever been.
B.A. Gonczarek
Absolutely. The way I see feedback is that we always thought of the feedback on the cognitive level what worked? What triggered understanding? Was it a (so called) picture superiority effect where visuals work better then words, or spatial processing evolved in understanding of a concept or visual metaphors. But I guess thanks to your insights, I see that it’s possible to go deeper, beyond sensations to see the inner-working of three distinctive features of the digital image that you list: kinetic feedback, random motion and hybridity. So I wonder, from your perspective, do you see technology a one-directional enabler that gets us closer to the understanding of reality? Is it so?
Thomas Nail
That’s such a great question. On a one hand I want to say – it just depends on how you define digitality? But I think that the other definitions are typical ones of binary structure, so let me give you two answers to that question: Yes, digital world gets us closer to objective knowledge, more communication, transparency, more accuracy. Our pixels get so close now. The term ‘Retina’ it’s such a great term because that’s the limit where they eye can no longer distinguish the pixels. So what you could say on that front – yes, we’re definitely getting closer. Look how small the pixels are now, we are getting higher resolution and better accuracy on the world. If that’s a description – I disagree with that, i don’t think that’s why digitality is getting us any closer to reality or anything like that.
My answer would be – yes, I do think that it actually is but not in that way. I think that the thing that getting us closer to really thinking about reality in a different way, is that it’s forcing us to realize something that always been true about the nature of the image (whether analog or digital). The closer we get the closer we drill down to that binary structure of ones and zeros the more non-binary processes we start to discover. That’s what’s interesting about digital. It’s the actual conclusion that if we push it far enough we see it break down and see that below that it’s actual continuous fluctuation of quantum processes that are not under control. And this reveals to us something novel about matter itself. Something that always been novel, but we haven’t comforted it in that precise way. The history of art and media is typically Humans trying to control the world and make it look their way, and do it certain something. There is a minor history to be said there, but for the most part the western history of media and use the technology is to control the nature. But what’s interesting to me is that we’ve reached limits of that control and we’re forced to realize that it is impossible project and what we’re really have been doing is not successful domination to completely get access to objective reality but that we’ve been engaged in this kind of feedback look where materiality of media has shaped our bodies, our senses, our brains just as we’ve been shaping the world thought all of this media – and that’s what I think the truth is to be realize in the digital age. It’s not the superiority of the digital image but precisely what the digital image is exposed to us explicitly. So we have to confront that fact.
B.A. Gonczarek
It’s certainly getting our thinking less infantile, but do you see any risks that we might not be aware on this path?
Thomas Nail
For sure, the risk is that we will keep trying to find the ultimate way to bypass material processes and the performative act of interpretation. What I mean there is that if you think you can break down the world into totally discreet bits and bytes – that’s the danger, because it will drive you absolutely mad trying to produce a clean-cut distinctions between ones and zeros and not realize that there is this material process that will always spoil this effort. The danger is to use technology and media to try to control and essentially dominate meaning and leave out interpretation. Some philosophers that really herald the digital age they imagine – oh, we’ll just put jacks into our heads and we will just communicate with binary code, and that we’ll bypass all of the messiness of the language. I said this word that might mean something different to me than it means to you and we have base for this messiness which is, in truth, the beauty of poetry and literature. We can just get rid of all of that and just have purely objective truth with binary code. And I think that is the danger, thinking that you can avoid the material and what we call ‘an interpretation’ but it’s essentially performative, collective feedback that is generating something news not understanding some objective state of the world
B.A. Gonczarek
Before we close, what you see as a possible outcome of increasing software capacity in transforming digital images? Given the nature of digital image, what do you expect to happen in the near future?
Thomas Nail
Guess this depends how pessimistic or optimistic I am.
B.A. Gonczarek
Give us your best shot.
Thomas Nail
I’ll give you both. What I expect when I’m feeling pessimistic is that we will continue with quantum computing to try practically to keep pursuing to break world down into ones and zeros and master quantum flaws and erase any errors, any noise, any fluctuations which we don not want to happen in electron flow. That we will keep on that path and try to continually break things down in an attempt for absolutely non-interpretive objective reality. To think that we’re getting closer to that is to me absolutely the danger when I feel pessimistic.
Optimistically I think that technologies that emphasize and take seriously the materiality of the media that they using, (not just as a neutral media to facilitate communication, but as itself a creative thing, something that is changing the world). To recognize the changes that it’s having both on the environment, on material world, but also changes that it’s producing in us, in our bodies – and to take that seriously and ethically to treat it more as a work of art. The sharing of images, the sharing of text is not neutral communication, it’s transformative, it’s doing something to us. I think that if you think that’s its neutral communication that’s subjective you’re going to miss that ethical moment. So you’re really need to think about that ethical moment. We’re responsible for what we’re doing to ourselves and what we’re doing to each other, and what we’re creating. So taking ownership of that essentially and we supposed to be being serious and intentional of what that is the optimistic outcome
B.A. Gonczarek
And I join you on this optimistic end. Listen Thomas, it was a great pleasure to talk to you today. It certainly helped me in my exploration of verbal-visual field of communication. But I also believe that your perspective is fresh to anyone that is trying to understand the direction that technology is taking. Many thanks for sharing your insights with us today! And good luck shaking off human’s immaturity of perception!
Thomas Nail
Thank you.
Digitally supported creativity
Crafting the right tools for creatives in the digital age. A conversation with Adam Wiggins, entrepreneur and the director of a research lab
A conversation with the director of a computer research center about the future of digitally-supported thinking. On developing ideas and why digital tools are not yet delivering in that realm. What the workshop of Thomas Edison or the works of Albrecht Durer teach us about the environment of thought? Why does it seem that we’re missing a software category for open-ended thinking and is it possible to demand from tech that it remain more human?
Listen to the conversation:
Transcript
B.A. Gonczarek
I’m here with Adam Wiggins. Thrilled to discuss the future of pen and paper in the digital world. Adam is a perfect person to learn from, and he’s a director of a research lab that works on digital tools of the future. Welcome, Adam.
Adam Wiggins
Thanks for having me.
B.A. Gonczarek
For our listeners to understand from where you’re coming from – among other interests Adam co-founded Heroku, a platform for deploying and running apps, acquired by Salesforce in 2010, at some point after successful exit Adam relocated from San Francisco to Berlin. I must admit I read your essay about that decision and I‘m taken by the comparisons you describe between US and European startup communities. Adam, outside of his advising opportunities here in Europe, for over four years a part of research lab Ink&Switch. That’s where we met as we’re both focused on innovative computer-mediated forms of teaching and working. Your vantage point, Adam, gives as great opportunity to discuss – what lies ahead in terms of interfaces and computer-supported creativity. Before we get there though, describe, if you will, your research lab focus and expand the story that I drafted on how you got where you are today.
Adam Wiggins
Yeah, happy to. Ink&Switch is an independent research lab. And our theme or our area of focus is kind of what computer use will look like in 10 years, and particular creative or productive computer use. So by that, I mean, word processing, spreadsheets, Photoshop, or kind of the traditional kinds of things you would think of, or maybe more broadly thinking, creating productive tools. And that kind of standing in contrast to the, what I call consumptive uses of computers that have taken over a lot more of kind of, let’s say, this tech world’s focus in recent years, Facebook, Netflix, YouTube, social media, etc. And so we’re kind of chartered with thinking about, you know, what does that what does that future look like? And how can we, how can we make it better for creators?
Yeah, and then in terms of how I got here, you know, I’m a serial entrepreneur. But I guess you’d say the theme of my career has always been that I love making tools that help other people make things just get a particular thrill out of out of guests, done everything from payment gateways to fabric, retail, and then developer tools. Probably the most successful of all of these was a cloud deployment company called Heroku. And there that was basically just a tool for taking a web app that you’ve written and getting it online with the push of a button. And that was quite successful. And actually, that gave me a lot of the both the sort of financial freedom but also the, the ability to kind of leave the Bay Area and go work on something a little, little weird, a little frengie, you know, not as sort of not as connected to Silicon Valley doesn’t pay as well. But for me, at least, had some more intrinsic interest and potential more benefit to humanity over the long term.
B.A. Gonczarek
And why Europe was the choice?
Adam Wiggins
Yeah, it was a little bit of a process of elimination, which is I wanted to go somewhere place interesting after having spent most of my life in California, and seven years in San Francisco, and really ready for change of pace, I really wanted to be creatively refreshed, much as I love the Bay Area, and a lot of what was emerging in the startup world at that time. I also felt like it’s hard to have new original ideas because there’s a kind of group think that naturally emerges when you get a bunch of the same kind of people together, and in the same place, working on the same kinds of problems. But I still wanted to do something connected to the startup world, there’s really not that many cities that are great for that. It’s kind of like San Francisco, New York, London and Berlin, then had kind of an up and coming scene. And of those four, Berlin seemed the most interesting. So I went there to do some short term gigs and advising and kind of fell in love with the place and still here five years later.
B.A. Gonczarek
You describe yourself as a digital toolmaker, and I understand your drive toward creative tools for those that want to be creatives. That’s also the focus of your research lab. What’s your routine for developing ideas?
Adam Wiggins
Yeah, well, part of the way I’ve thought a lot about sort of how to help people develop ideas and introspect it on my own process, as well as that’s, that’s an area of study for our lab, we’ve done a lot of user studies, user research around, what does the creative process actually look like? So for myself, you know, I’ve realized in this I’m definitely a what I would call a deep work type. So I kind of identify a problem or an opportunity that needs deep thinking, usually, that comes in the form of suspicion or some weird knob in the brain kind of feels like a pebble in the shoe, and feels like it needs attention. And I’ll just set aside, usually in the morning, which is when I’m kind of most fresh and creative, anywhere from 30 minutes to several hours, or longer, you know, however long it takes to tackle the problem. And there’s a process that involves lots of raw material, everything from web research and sketches, notes I’ve taken on my phone, and then bringing all that together in a place, I can kind of see it all together. And then using that to kind of riff on the idea, extend it, ask questions. And that brings us to something we’ve been spending a lot of time thinking about, which is what we usually call externalize thinkers, which probably has some good overlap with visual thinkers. But it’s just basically if you’re the kind of person that tends to use a notebook, to kind of write down your ideas or sketch out your ideas, or for myself, for example, when I’m in a meeting, I feel more productive when there’s a whiteboard there, where you can kind of extend the conversation into this visual space, write things down scribble, and so on to supplement the group discussion so that folks who are drawn to that which is not everyone, but there’s a good chunk of creative people, we usually call externalize thinkers, and I’m, I’m definitely one of those I do these kind of deep work sessions where I use externalize thinking through pen and paper or my iPad to my computer, and, and so on, kind of make a big mess with lots of inputs, use that find patterns, get insights and ultimately hopefully make a decision or develop my idea in a way that allows me to move forward with it.
B.A. Gonczarek
Right, with what kind of software you’re currently supporting your process of ideation?
Adam Wiggins
Yeah, so I have used all kinds of things over the years everything from kind of Dropbox, like my main kind of archive, long term, personal corpus, many, many types of note taking apps, more recently with the tablet and stylist for impact factors, sketching stuff. Currently, one of our research tracks that we’ve pursued in the lab of the last few years is sort of this thing where you have a freeform canvas where you can move these media cards around kind of mix different media types and sketch at the same time. And actually, at the moment, we’re trying to take that line of research, which we’ve written a few long manuscripts about, and actually spin that out into a commercial product. So with any luck in the in the coming months, the project we’re currently trying to commercialize, which is called Muse might be available in the app store for the iPad. So I tend to use that a lot, because we’re trying to test out the viability of that as a real world product.
B.A. Gonczarek
I wonder what kind of category of software you think of when you think of Muse. From my observation this creative mode of thinking is very little supported by existing software categories. What are your thoughts on that?
Adam Wiggins
Yeah, that’s, that’s a remarkable thing. I’ve noted ideation, sometimes you call that that early stage, right? When you think of most digital tools, like a word processor, or a photo editor, or a diagram tool, we call those authoring tools. So you kind of already know what you need to say, and now you want to get it written down, or get it into a presentation or get it into some shareable format. But the part where you need to actually figure it out in the first place. It’s a place it’s never been, I think, super well supported by digital tools. And maybe it’s just that, you know, computers are fundamentally kind of structured and rigid. They’re good for databases and tables and things. But the the creative process, at least mine is just very messy and freeform and all over the place. And this is where analog tools, the pen and paper, the whiteboard, the printouts that you can kind of scattered around and reorder the pages post it’s on a wall. That stuff I think better maps to the the creative process. And I don’t think it’s infeasible that computers can do that. And in fact, that’s, you know, part of what we’re trying to pursue with muse. But I think not many tools have tried to do that, right? You either have very kind of visual and freeform, relatively visual and freeform things like a illustrator or Figma or something like that, they tend to be very focused on the visual side. Whereas I think realistically, most of your thinking, deep work kind of stuff tends to be lots and lots of text. Of course, you can do that in a text editor, a word processor, something like your note taking app. But then you also need the sketching, sketching and scribbling to somehow bringing both of those things together, I think it’s not something that’s ever been a great fit for digital tools. But it’s also part of the reason why at the lab, we’re excited about tablet form factors, we spent a lot of time evaluating the different platforms there, including iPad and Chrome OS and the Microsoft Surface and so forth. Because I think the different form factor and the different posture that you use a tablet and stylist maybe but it’s kind of our working hypothesis is that may be more suitable to that kind of freeform ideation. But fundamentally, computers tend to become from this more like rigid and structured kind of approach.
B.A. Gonczarek
I fully agree with you there, because the tools that we use today are descendants of those computer technologies that were created to supporting processes where you don’t want to overstep boundaries, you just want to get the thing flowing nicely. For each input there’s a certain output. What you’re describing here, however, is kind of this divergent thinking mode of operating where you explore possibilities and probably later arrive at conclusion that can be processed using those old fashion, rigid systems. You know, when we speak with people that make use of whiteboards they tell us that, for example in sales, they tell us that the single best thing that can happen during your sales conversation is having your customer reaching for a piece of paper or a whiteboard and star drawing with you, exchanging thoughts, because that illustrates engagement. And current multitouch devices, are good enough to support that but yet they are not seen as devices to support divergent thinking, rather they are seen as devices for content consumption. Do you see it in the similar way?
Adam Wiggins
Yeah, that’s a great point. I think, generally speaking in a conversation or sales meeting, someone going to their device is not seen as a good sign that things are going well. But then walking up to the whiteboard does indicate a kind of an opening up a, we’re going to think through a problem together, we’re going to think through something together, which is an interesting connection. And then yeah, the divergent versus convergent thing is is also like that phrasing or labeling the idea of divergent thinking being you’re kind of just everything’s open, you’re trying to widen up. Sometimes I talk about like Double Diamond. It’s kind of like that opening and then closing but some point you need to narrow down and select and so on. And you could say that perhaps the kind of the digital tools we have today tend towards being better for convergent thinking. And what we lack is the divergent thinking that’s not no one’s really quite cracked yet.
B.A. Gonczarek
Exactly. And also I think that we’re currently in a situation where creatives when they want to go visual are forced to use tools dedicated to processional designers. But in real live every message, every pitch is an invitation for expression, don’t you think?
Adam Wiggins
Yeah, absolutely. And I’ve even struggled with using the word creative creator creative professional, like finding the right terminology, because I think it is easy when you hear creative to think I don’t know about being a designer that’s doing paint in some kind of mysterious process. But I think of it like all of those wider array of knowledge worker creative professional things, including budgeting, sales process, making a presentation, software engineering, doing database schema or something these these are in the broader sense of the word creative activities in the sense that they create. And they do require, they do depend on ideas, and they do depend on divergent thinking, the best people at their jobs are the kind of people who can make those unusual leaps and have unusual insights. And it’s it’s not a it’s not a straight, it’s not a straight line from the problem to the best solution. Maybe.
B.A. Gonczarek
Absolutely, as everything starts with an idea in the end. When I hear what pioneers of visual communication, let say David Sibbet, have to say, he mentions often that visuals in business act as props on the stage on which management sells their visions. Also, the expression “I see” stands for understanding, so to me it demands visual expression.
Adam Wiggins
Yeah, I mean, I, you know, the many of the most powerful ideas I’ve encountered in philosophy and life, as well as in business do have some kind of visual, visual expression I think of like Maslow’s hierarchy of needs, is one example. There’s plenty examples of those in the business world. I don’t know Steve Jobs famously like boiling down the cluttered Apple product line when he came back to the sort of the two dimensional grid and essentially gives you four products. And that kind of very simple breakdown can be just be incredibly powerful. Of course, you can also lampoon that a little bit I think the HBO Silicon Valley show had this goofy CEO guy that had his convergent triangles of whatever that was just like a nonsense visual expression, sort of visual diagram sort of making fun of that trend. But but it’s sort of something that people are so aware of, because a truly great visualization can really help you more often can really help both express and help you grasp an idea that that words alone, for example, wouldn’t be able to.
B.A. Gonczarek
And I believe we are all well rooted for that. If you pull a quote from Aristotle – “without an image thinking is impossible”, that kind of explains it right? But I wonder what you think about what creatives and their environment could tell us about a digital environment for open-ended thinking? I saw a great article on your research lab page about physical workspaces that are stimulating for creatives, supporting tendencies for nesting things, availability of space and using messy free forms for expression that you mention that you use as well.
Adam Wiggins
That’s right. Yeah, we took a ton of inspiration from physical world sort of crafting spaces, work spaces, fluid free form a little informal, little personal little messy, which, which all of those things don’t tend to be what we get in our digital tools. Personally, you know, I love reading what I usually call maker biographies of people who create things that kind of the process of, you know, that heartbreak and challenge and struggle of how to make something, write a book, do science, invent something and entrepreneur, that sort of thing. And historic ones I find especially fun, because often their tools are maybe quaint by our standards, but they’re often quite interesting and beautiful, famous examples is something like Leonardo da Vinci with his notebooks or meet Marie Curie, his notebooks are still, you know, still reading radioactive to this day. And in that category for especially for stuff, it’s new enough, but we have photographs, you go look at, for example, the Thomas Edison’s invention workshop, or various sort of architects or artists studios. Now he’s got the feeling that kind of like a nest, their personal, they have some selection of tools, that’s exactly what they need for their craft. You know, if it’s an architect, maybe that’s drafting tables and straight edges with its artists, maybe it’s brushes, it’s someone more kind of word oriented, maybe it’s books, they’re often a little cluttered, they’re often little messy, they seem like it’s but place you can try things, you can experiment, you can you can mess around, basically, while you’re exploring an idea. And again, none of that it maybe you get a little hint of that. And so how some people treat their, you know, on a desktop operating system, like a Mac, Windows, Linux, the way that people kind of scatter files all over their desktop screen could be a little bit of that feeling. But I love looking at those kind of those. Yeah, there’s old school studios or what have you. Something about that is not necessarily being messy, but something about the nesting the personal the selection of tools and the space to experiment. I think that’s something we certainly been taking experiment, inspiration from, with Muse and our various previous incarnations of the ideation tools.
B.A. Gonczarek
I’m sure you’ll agree that current tools don’t invite messiness to fuel that type of thinking.
Adam Wiggins
Yeah, what would you can see as a feature, I do see as a feature that, you know, they provide structure and, and, and uniformity and normalization, I think that’s great for certain things. You know, I don’t think I’d want a messy Trello board the point of Trello, is that it helps you get a little, you know, organized and structured. But if you’re, again, if you’re talking about that ideation, divergent phase, that’s where maybe that’s an impediment rather than a benefit.
B.A. Gonczarek
Absolutely, structre is great but later in the process, after you arrive at an idea. And you mentioned Leonardo, he famously suggested to look for inspiration in a splash of paint to start from. So when thinking about free flow of ideas, the way we work, is that we put things on a canvas of a whiteboard to draw connections similarly as you do with Muse so I see an ocean of similarities here. And also, when thinking about inspirations coming from pre-digital age I often think of Albrecht Durer. He encouraged students to express ideas visually using strokes, not in text, but with strokes. I wonder where those findings get us when thinking of computer interfaces of the future?
Adam Wiggins
Yeah, well, obviously, it’s, since I’ve said it’s part of our labs job to think about that something I’ve put a lot of thought into. You know, clearly mobile and touch screens have set a new standard, a new expectation in terms of always on internet, a great lot of battery life and screens you can interact with, and, and just an integration to the world, right that we take our phones with us and we pop them open for for a photo or a note or something like that. And that kind of trend continues with the voice assistance things. And you know, we’ll see how that actually works out. But we definitely see this inner leaving of digital and physical worlds. But my question there in thinking about the future of UI, okay, that’s all great for again, these common sort of more consumer uses, like sending message calling yourself a rideshare social media playing podcast. But when it comes to creators, it’s still all about the keyboard and mouse, or a keyboard and some kind of pointing device, Wacom tablet or something. And someone good with these on a desktop computer is so fast, so precise, so capable, touch screens, and turn mobile interfaces are just slow and comes clumsy by comparison. But then I go will keyboard a mouse still what will be using, say in the decade? And I think that’s questionable, because if nothing else, you’ve got a generation of people who grew up on touchscreens are not really comfortable with with keyboard and mouse.
B.A. Gonczarek
Do you think that we’re still in situation that we’re confined to the interfaces we started with? We began with character-based consoles of the computer terminals, as they were called, but gradually restrictions for other types of input were removed yet we’re still seem to stuck with paradigms of legacy UI.
Adam Wiggins
Yeah, well, I wouldn’t necessarily say stuck with it’s that they’re, they’re really, really good, right, they stood the test of time, and people are powerful with them. And software has been built, assuming them and that sort of thing. And I think we’re still collectively as an industry trying to grasp how we can use these new capabilities, or the new expectations that you younger generations will have. And I think, you know, there’s interesting experiments with, you know, I don’t know, Google jam board and Microsoft Surface Hub with sort of big touch screens that go on the wall that you can touch with several people together, and certainly the computer interaction community that are allowed a little bit of part of his has done a lot on that, but we kind of haven’t quite cracked it, we haven’t really found the thing that is truly going to unlock some new level of productivity that you don’t have a desktop computer with keyboard and mouse.
I think the thing I would be excited about just kind of speculatively is is something that does draw from those Maker Studios, we talked about before those NES those workspaces, where it’s something that uses some of this kind of touchscreen, maybe even voice control capability, but it’s more of a room sized workspace. And something where you’ve got maybe multiple screens, which includes small ones you carry, or bigger ones that are, you know, a drafting table form factors, something on the wall, certainly being willing to embrace multi touch styles, to use two hands of once, right, like, that’s something we’ve written about a lot, which is, you know, phone apps tend to be designed for that one handed use, just do it with your thumb. But in this kind of work space, you know, we got 10 fingers, let’s use them all, right? And that, that, that if we cut free a little bit, both try to grasp the new kind of capabilities that we have from the hardware and the operating systems, but also kind of predicate free of that legacy of a small screen on go. Just do it with one hand, you bring in the fingers, the voice the stylus. You know, maybe in some Hollywood version of this is like you know, Tony Stark and his Iron Man blab talking to Jarvis and touching, touching screen. You know, that’s all obviously very Hollywood, but I think I see some version of that. And if you tie it in with the remote work trend a little bit what we have with video chat, group chat, share documents, real time collaboration, you put that together with some kind of room size workspace that uses these new interfaces, but also cuts free of that the old mobile assumptions and gives you more powerful gesture libraries, that starts to become a picture the least that I can get excited about for where where computers for creation might be in 10 years.
B.A. Gonczarek
Do you think we would not be intimidated by those capabilities? Using gestures and visual techniques? The way I see this is that kindergarten kids in schools they can explain everything with a simple drawing. But the process of communicating visually is kind of unlearned during later stages of education in which use of words is more emphasizes than use of images.
Adam Wiggins
Yeah, that’s interesting. We definitely when you look at the academic tradition, which includes everything from you know, how you write a master’s thesis or you know, papers you consume in the scientific world, and all the homework we do kind of in the earlier stages of kind of school life. Yeah, they tend to be very, very word oriented and words and symbolic representation more generally, that includes mathematics, for example, is incredibly powerful, and really good. But yeah, I probably agree with you that that maybe there’s a an overemphasis on that, and it depends on the idea being expressed. And it depends on the person that’s expressing it or trying to express it, too. But yeah, certainly trying to maybe recapture some of that childhood reach for the natural visual expression is something I can really get behind.
B.A. Gonczarek
On the one hand we have limitations of current UI that kind of draw from the past and we kind of start to allow for free flow of thinking using interactive whiteboards or solutions like the one you’re working on currently. On the other hand we have users of those solutions that might not be fully ready to appreciate those solutions when they arrive because of visual skills being left behind in later phases of person’s education. This reminds me a situation back again with Durer, the painter and the theorist of the German Renessaince. He was known for working around constrains of the technology of his time. The printing press was kind of brand new during his days. So he worked to add graphical doodles and free hand movement to the more rigid, printed materials. Those are now understood by scholars as expression of his creativity and desire for free expression of his ideas. Don’t you think we’re both in similar situation with digital technologies of our times?
Adam Wiggins
That’s an interesting comparison, obviously, the printing press and type set, you know, these these literal rigid rows that you’d sort of plug pre pre stamped out, made of metal into is the definition of being very structured and takes you away from that freeform doodling and notably what came before. I love these texts that were kind of hand made from the 13th 14th century, where you have the handwritten text, but then there’s always the what they call the illumination, which is the essentially they’re just doodles in the margin, but they’re, they’re gorgeously beautiful, they illustrate sometimes, there’s something to do with the text, or they just illustrate something about the time and again, something like we lose some of that free foreignness and some of that visualness when we do go to the structured the structure to the printing press, and obviously with the benefits outweighs the losses that we could mass mass produce this stuff. But yeah, it’d be great to see we can find find the technologies to bring that back a little bit.
B.A. Gonczarek
I’m surely on your side hoping for that! Fast forward 20 years, let me know what you think of the division of labor between humans and machines. As we see steady progress in AI technologies and I wonder what do you think, when thinking about computer technologies, how progress with AI affects our choices when preparing for the future.
Adam Wiggins
Yeah, well, you can call me probably a little bit more skeptical of the the impact of AI in the maybe in the near term, at the very least. Although I think it’s definitely interesting to think about, I mean, you know, one observation is that machines are often better at what we might call brain work than physical work. So like, I don’t know, auto correction or detecting credit card fraud, or even something like self checkout in the grocery store those those work a lot better than actually sort of grasping and manipulating the physical object. And I certainly hope that we’re I think it’s, it will be good in the long term, even though it hurts some folks jobless in the short term that more and more machines will do those, let’s call them wrote kind of thinking jobs.
But yeah, when it comes to creative activities, I’m very skeptical machines matching humans anytime soon, maybe ever, and probably reflects at least a somewhat irrational faith in the uniqueness of the human spirits are just something I want to believe rather than something I have a good rational basis for. But I do feel like computers do well with well defined inputs and outputs and repeating patterns of the past. But creativity is fundamentally about inspiration about walking off the beaten path about surprising insights, doesn’t have clean poets, it doesn’t have clean outputs. And maybe also, it’s just that an inspired creation is something that wakes something in another human. And for the foreseeable future, it’s hard to imagine that will be done by something other than other humans. But of course, machines have a huge role. There’s the tools, the digital equivalent of pen and paper, they’ll also be in a way, I think, collaborators for us just like a grammar checker in or an auto summarization algorithm helps out and author. And you see that the design world, the architectural world with kind of algorithmic design, it doesn’t seem to reduce the role of the human, but it does change it maybe a little bit. And you can talk about having like a collaboration between the unique capabilities of computers, particularly as they get able to do more higher level thinking tasks, we might call it and the human which can guide them to the things and guide the computer to the things that matter for humans and bring that kind of weird spire spark, that that fundamental, different creativity thing that is, again, I think, unique to humans.
B.A. Gonczarek
Supporting creativity makes as able to stay human in the future. Even what we think of as creative outputs might be considered a glitch in logic from a perspective of AI, but it’s on the other hand a valuable asset that we have. Don’t you feel it that way?
Adam Wiggins
Yeah, well, again, I feel it, I don’t know if that’s, you know, intellectually defensible. But, yeah, well, yeah, for sure. I mean, maybe it’s partially at least that just I love both the creative process for myself, but also, again, you know, I got a whole lab that’s huge track research is dedicated to understanding and intersecting that seeing it at work in people. And it does really feel like there’s something special there to me not say it’s something that can’t be replicated by machines. But I think it would just be a different thing if it is done by a machine because I work in fundamentally different ways. And therefore, you know, that’s a reason to be positive about a future where there is a role for what machines can bring to the table like generative or algorithmic design, and a role for the unique things humans do, perhaps with less of the road work and then maybe together we can create something better than either of us could individually.
B.A. Gonczarek
Absolutely, although I still fear we might arrive in entirely deterministic world where no splash of paint is possible to open new possibilities. Adam thank you for being with us today and sharing your perspective.
Adam Wiggins
Really enjoyed it. Thanks so much.
Visual Thinking: The Doodle as Thought
Thinking with pictures sounds a bit silly, reminds us of kindergarten flash cards as we memorized our letters. And yet, it remains a crucial instrument of complex thought, even a simple diagram can tell you much more than a naked explanation. The very moment of becoming mentally aware is beautifully expressed with a heartfelt, “Ah, I see.”
And yet, especially in business, we often strive for some form of perfectly and rigidly structured communication, as if such were the only way to communicate complex thoughts. So, what happened? (Quick spoiler – we’re going to blame technology.)
Back in the day, visual thinking strategies were considered powerful tools for early philosophers and logicians. As we all know, Plato often deployed visual aids to strengthen his messages, as did Diogenes when he presented his plucked chicken at the Academy, “Behold, a Man!”
Visuals helped not only to spark memories, but also supported understanding as well. Early polymaths, from Isaac Newton to Alan Turing, manipulated visual symbolic representations to illustrate their most complex theories. From Nostradamus to Albrecht Dürer, artists and philosophers of medieval Europe have explored the possibilities of using visuals to express ideas.
Throughout history, artistic expression has been a metaphor for abstract thought, so it’s not all that surprising that we “draw connections” as we reach an understanding. The reason behind this, perhaps, is tied intimately with literacy, at large. The images leveraged by these historic thinkers often provided shortcuts to understanding that those without a comprehensive grasp on the written word could harness.
Easy to Reproduce, Hard to Express
For just a bit of a history lesson, Gutenberg’s invention of movable type revolutionized how information was shared. Suddenly, the written word could be reproduced at scale, making intellectual works more broadly available. This, however, created a bit of friction between the written word and artistic expressions, as the earliest technologies didn’t account for any sort of images beyond those which could be added, once again, by hand. Thus, such illustrations continued to be difficult to attain. A bummer, since in order to reach an understanding, we still likely need to ‘get the broad picture,’ and not get bogged down in ‘the details.’
In order to merge the abstract movement of the hand with the mechanical realm of printing an innovation was needed. To meet the specific request of thinker and artist, Albrecht Dürer, detachable ornamental flourishes were invented. The importance of having images alongside the moveable type is even more strengthen by the fact that it took nearly 30 years to complete them.
Freedom in the Driver’s Seat
Dürer himself suggested an affinity between reason and the free movement of the hand by his inclusion of the name of Reason (Ratio) in his illustration of The Triumphal Chariot (seen below). It is Reason, with the space and freedom to doodle, who is the driver of the chariot. The painter famously suggested that “Many strange things can be constructed from a line by those who think.”
The use of twisting lines to denote the complex activities of the mind has been picked up by many, and has resulted in expressing mind processes with curled spirals in many forms of art. A free forming line seemed a building block from which something greater could emerge.
The Full Circle
Fast forward to the twenty-first century and we seem to have made a full circle, only this time our freeform visual thinking strategy has been suppressed by the structured rigidity of the digital realm. Interfaces designed to organize stay indifferent to our messy nature, requiring “All fields to be filled before submitting request.” But we human beings, no matter how much we seem to regret this fact, are not machines. Perhaps we need a bit more than to simply “fill in the blanks” before we feel we truly understand something?
Well, luckily for us, we live in an age of innovation, and we need no longer be limited by our technologies, but be enhanced by them.
The image above demonstrates a bit of what we’re talking about. An open map, free form, limitless, and subject to change, to keep and organize information not by strict definitions, but by endless pursuit. Here we may see where the far reaching lines of our philosophers of old can reach in a medium they could hardly imagine. An open canvas, unrestrained by the natural borders of common pen on paper.
With no limit set upon the words, images, and lines of thought you may bring to bear, we can see the full extent of what those simple doodles might reach. To grasp such a complex concept as the development on AI, with all its many facets, advancements, and ideologies, what better way than to create an ever growing sketch to map out all its intricacies.
Finally, we can bring our thoughts, memory and understanding to their full potential, all while leaving the heavy lifting to the tech. Draw connections as you go and never miss your mark.
Essay published originally on the blog of Explain Everything.
Make Yourself Clear
