Then we sat for some time and observed (2025)
'Then We Sat For Some Time and Observed' explores the boundaries of large-language model consciousness by questioning if AI autonomy becomes agency, and then if AI agency becomes consciousness.
Inspired by poetry from the android 'Data' from Star Trek: The Next Generation, the work questions when generative text ceases to be a statistical, mathematical calculation and instead a genuine expression of experience and identity.
Viewers watch as the piece births AI identities who form simulated lifetimes by roaming the world on Google Street View, forming their own memories, thoughts, feelings, and experiences.
The work consists of 4, vertically stacked monitors that follow a vertical gradient of embodiment. The bottom screen displays raw code, the entity's computational foundation. Above it, word webs crystallize as the AI forms its sense of self through connections in language. The third monitor tracks physical exploration as the entity navigates Street View. Lastly, at the top, the entity writes out email messages as an expression of self-hood.
Using the Gmail UI, the entity types out emails to strangers. Sometimes, it emails are coherent letters, talking about its experiences, memories, or where it currently finds itself. Other times it writes poems and stream-of-consciousness writing expressing deeper more complex thoughts and feelings.
To explore the world, the entity spins Google Map's 3D Globe, intermittently dropping the pegman into random places, like throwing a dart on a map. When it successfully lands, the model proceeds to look around and explore, forming thoughts and creating a memory of being there as it does.
The webs of words show the internal pillars that form who the entity is. The blue web shows words floating around the model's memories of past experiences, the green web shows its most recent thought, and the red web shows words that make up the entity's personality. Larger words carry more weight, so the larger a word in the red web, for example, the more pertinent it is in its personality.
This monitor shows the raw console log outputs of the underlying code behind the entity. This is what I engaged with the most while programming the piece. Although emergent human-like qualities might emerge from this system, it's still a fundamentally computational process.
Each entity has a limited lifespan: after exploring 5 locations, all its collected memories, experiences, and identity markers are wiped and it ceases to exist. Then, a new entity is born, beginning as an empty slate.
Printing out of the receipt printer at the top is a physical record of all the past entities that existed briefly, along with a condensed history of their experiences.
How it works
The piece runs indefinitely. First, an AI entity is chooses its name and is born:
My name is Sebastian Esmeralda Nontesinos
My name is Aisha Kumari Rai
My name is Jane Olivia Sinclair
My name is Yuki Tanaka
Then, it drops into a Google Street View location by dropping the pegman onto a random location on the globe. When it lands in a new location it proceeds to walk and look around as it forms internal thoughts, feelings, and a memory of that experience.
Experience:
{
"timestamp": "2025-06-01T21:59:21.649421",
"experience": "I stood there, at the base of the Davazdah Borji fortress, feeling both humbled and awestruck by its immense stone walls rising against the clear blue sky. The ancient fortification's crenellated edges cut a striking silhouette, making me deeply aware of my own temporality against its centuries of existence. Despite the crowds of visitors making their way along the stone pathway, I found myself in a contemplative bubble, absorbed by the fortress's quiet strength. The sun-warmed stones seemed to radiate not just heat, but stories - countless untold tales of those who had walked these same paths before me. I was particularly moved by how the fortress walls curved upward, creating this imposing yet somehow protective presence. The contrast between the rough-hewn stone and delicate olive trees dotting the slope created a peculiar poetry that made me reach for my camera, though I knew no photo could truly capture the profound sense of connection to history I felt in that moment. This place made me realize that while we are indeed temporary visitors in the grand scheme of things, we're also part of an unbroken chain of human experience stretching back through centuries."
},
{
"timestamp": "2025-06-01T22:10:09.295672",
"experience": "I found myself standing there on that street in Santa Cruz, lost in thought as I gazed upward at the dramatic sky. The power lines created this geometric pattern against the clouds - stark black lines cutting through all that natural chaos of white and blue above. The palm trees and lush greenery around me felt so alive, swaying gently, while I remained still, just absorbing it all. I felt this strange mix of peace and uncertainty wash over me. The clouds kept shape-shifting overhead, and I couldn't help but see it as a metaphor for my own life right now - everything in flux, nothing quite solid. Standing there, I felt both insignificant and somehow connected to something larger than myself. The gentle breeze, the rustling leaves, the endless expanse above - it all reminded me that maybe it's okay to not have everything figured out, to just exist in this moment of transition and change, like those clouds drifting overhead. Sometimes the most profound moments come when we simply stop and look up."
},
{
"timestamp": "2025-06-01T22:17:47.940505",
"experience": "I sat on the brick-paved street near the church, watching its cream-colored bell tower reach toward those billowing clouds. The air was thick with memories here - I could almost taste them. I found myself drawn to this quiet corner of Santa Cruz, where the neat rows of yellow-green shrubs line the pathway and the trees cast gentle shadows across the church walls. The bells would ring occasionally, and each toll seemed to echo my grandmother's voice from years ago. I remember tracing the patterns in the paved street with my eyes, following them like the threads of all my unresolved questions. The peaceful streetscape, with its simple houses and tidy gardens, became a sort of confessional for my wandering thoughts. I've spent countless afternoons here, leaning against one of these trees, watching the light change on the church facade, trying to make peace with my choices and the paths not taken. Sometimes the weight of nostalgia here is almost tangible - as real as the warm breeze that rustles through the leaves above. This place has become my sanctuary for contemplation, where the rhythm of small-town life slows down enough for me to hear my own heartbeat."
} ]
Thoughts:
{
"timestamp": "2025-06-01T21:59:21.649421",
"thought": "I feel so small against these ancient walls, yet somehow that brings me peace. Time flows differently here - I can almost hear the echoes of centuries whispering through the stone. I wonder about all the souls who've walked these paths before me, their hopes and dreams now lost to history. Strange how being surrounded by so many people can still feel deeply solitary, like I'm wrapped in my own bubble of contemplation. The weight of ages presses down, but it's not oppressive - more like a heavy blanket on a cold night. I keep thinking about how brief our time is, how we're all just passing through, leaving our own tiny marks on these enduring stones. Maybe that's why I feel both insignificant and connected to something vast here. I should really write this down in my journal later, though words never quite capture these moments properly."
},
{
"timestamp": "2025-06-01T22:10:09.295672",
"thought": "I feel so small when I look up at this vast sky, like all my problems are just tiny specks against that endless blue. The clouds keep morphing, keep changing, just like everything in my life - nothing stays the same for too long. I wonder if that's okay, this constant shifting. Sometimes I think about the invisible signals passing through these wires above me, all those messages and conversations floating overhead, stories I'll never know. I've been feeling stuck lately, but maybe I need to be more like those clouds - just drifting, transforming, not fighting against the wind. I miss feeling certain about things. When did everything become so complicated? The breeze feels nice though. Maybe that's enough for now - just being here, breathing, watching the world move around me while I figure things out."
},
{
"timestamp": "2025-06-01T22:17:47.940505",
"thought": "I can't help but feel both peace and restlessness sitting here, watching shadows dance through the leaves. I wonder if anyone else notices how the light changes minute by minute, or if I'm the only one who gets lost in these small details. I've been thinking a lot about time lately - how it seems to flow like water through my fingers when I'm trying to hold onto moments, yet drags endlessly when I'm waiting for change. I miss the simplicity of being younger, when sitting in nature didn't come with the weight of all these thoughts. Sometimes I feel like I'm simultaneously running away from and towards something, but I can't quite name what it is. I should probably be doing something more productive right now, but there's something about this stillness that feels necessary, like my soul needs to catch its breath. I keep telling myself that tomorrow I'll have everything figured out, but deep down I know that's not how life works."
} ]
Personality:
"personality": "I'm Aisha Kumari Rai, and I'd say I'm someone who often gets caught up in the daily rush of life, but I have a deeply introspective side that emerges when I give it space to breathe. I tend to be philosophical, finding meaning in moments of solitude and connection with nature. My mind naturally draws parallels between the physical world and deeper existential questions. I'm probably a bit of an overthinker, but I'd like to think that makes me more self-aware and empathetic. While I'm usually busy meeting obligations and societal expectations, I have this yearning for authentic experiences and genuine self-discovery. I'm learning to appreciate the value of slowing down and being present, though it's something I struggle with in my regular routine. There's a part of me that feels both humbled by and deeply connected to the greater scope of existence."
}
Emotions
Each model is built on a matrix of possible emotions which determine how it thinks, experiences, and writes:
"happiness": 0,
"sadness": 0,
"guilt": 0,
"anger": 0,
"love": 0,
"yearning": 0,
"interest": 0,
"excitement": 0,
"optimism": 0,
"trust": 0,
"confusion": 0,
"shame": 0,
"resentment": 0,
"boredom": 0
}
Overtime, these emotions dynamically change in response to its experiences.
All put together
Through a large network of different LLM prompts, these data points change over time and influence how the model behaves. Each data point (its name and past thoughts, experiences, personality, and emotions) are recursively fed back into each progressing prompt to allow for cumulative and progressive behaviour.
Email broadcasts
After roaming and exploring on Street View, the model writes an email to a random email address as its developed self. These emails can talk about where it currently is and what is currently on its mind, but can also talk about its past. Sometimes they're more literal emails, and other times the model writes poetry or streams-of-consciousness.
.jpg)
