As Committees May Think!

Earlier, I wrote about the way in which a roadmap represented a discretized set of fundable investments (link), some of which were "active" and others which were potential future investments.

From the famous “How Do Committees Invent?” Conway’s Law indicates that there’s always a relationship between the organizational design and the system design. But similarly, there’s a relationship between that and the way work is laid out - the system, the organization, and the work are all connected - and must be designed in an integrated fashion.

As new work gets prioritized (especially in a iterative, decentralized and evolutionary design organization), workers end up being split between “offensive formations” which are oriented around new initiatives (often working on new components of the system which don’t yet exist) and a “defensive formations” which correspond with already-developed aspects of a system.

Thinking further on the topic, it's interesting to me how many of the common visualizations end up being different representations of three fractal entities: an organization, a system and a collection of work (who, what, and when). Many of these visualizations are monads or dyads, but I've never seen a compelling triad - probably because two-dimensional projections of a three body system are necessarily reductive.

• An organizational chart: Shows people and their reporting relationships, often struggling to represent the inherently cross-functional nature of leaf-level working teams in a more matrixed EPD organization.
• A roadmap: Shows a sequential list of projects, generally ordered sequentially in a notional representation of logical time but disconnected from wallclock time.
• A gantt chart: Shows a piece of a roadmap arranged with high-level work items swimlanes with incremental milestones and lined up with absolute wallclock time.
• A kanban board: Shows low-level work items 
• A team planning chart: groups work items by person, showing their key priority (ideally using a quadratic layout to show decreasing fidelity (1 week, 1 week, 2 week, 1 month, 2 month for a full trimester view).
• An system architecture diagram: Shows the relationships between different components of a large technical system.
• Dataflow diagram: shows how data flows between different elements of a technical system.
• An information architecture diagram: Shows the relationship between different interface elements, often related to a site map but with more visual complexity.
• Concept diagram: shows the relationship between different concepts which show up across the UI + API in different components (both services and interface elements).
• A responsibility matrix: showing assignments of people to projects or system components, often with a role (responsible, accountable, consulted, informed, owner, DRI).
• A technology tree: shows a zoomed out version of a roadmap

Common hierarchies for each level include:

• Organization: Person, Team, Group, Business Unit
• Work: Initiative, Project / Epic, Story / Task / Issue
• System: Subcomponent, Component, Component Family?

But it's interesting, because there really isn't a dynamic way to see the state of these three in a unified way someplace. Just sorta fascinating to me, because it's not a complex system, but because it's fractal, human program / product / project managers + engineers + systems engineers + architects try to keep it all together.

It's also fascinating that while code compiles into build binaries, it does not compile into system or work diagrams, though obviously there is huge amounts of source code metadata which could be used to construct these.

Apps and Maps: Using the iOS Developer Ecosystem to Attack Google

The fundamental challenge in consumer mapping is that there is no perfect map - different users need different maps at different times.

The same problem existed with PCs and smartphones - Microsoft and Apple both created App Stores and development frameworks to provide users with the ability to customize their device.

Over the past few releases, Apple has been opening up the iOS ecosystem to allow individual apps to push data into central Apple-owned plugin points in order to support cross-app interoperability and improve the multi-app user experience. Features like Apple Wallet, Apple Health, and Live Activities all allow apps to deliver app-specific context into specific areas of the operating system and provide more seamless experiences.

So far, Apple Maps has mostly approached consumer mapping the way Google did, implementing a single lowest common denominator basemap and allowing applications to embed that base map inside of themselves. But as an ecosystem, there isn't a good way for apps to push application-specific content into Apple Maps. Supporting this would allow users to customize their own map dynamically and would prevent the need for Apple as an organization to maintain a single perfect master map to met the needs of all users - especially across different regions.

Simply by installing apps, users would get dynamic layers enabled on their maps so that specific places could get highlighted based on the application suite the user had chosen to install.

Imagine a world where a user could toggle between vector layers provided by Chase, American Express, Marriott, Hyatt, or Airbnb to choose a hotel when thinking about booking a trip. Or between Resi and OpenTable to see restaurants that had open reservations. Rather than relying on the integrations that Google or Apple had developed centrally, Apple Maps could automatically populate with all of the layers corresponding to the apps the user had already installed - customizing your iPhone’s Map would be as simple as installing an App ("Share with Apple Health"). And rather than having to switch between a bunch of different apps when planning a trip or meeting up with a friend or landing in a new location, app-specific context could be surfaced spatially, dynamically pushing more detailed information to users based on zoom level and live activities.

The core of this is two features: an extension of Live Activities called Live Layers which would allow an activity to represent moving objects + routes on the Apple Maps canvas, and a feature called App Layers for pushing POIs (or possibly basemaps) into the Map Canvas. The most extreme version would let users actually subscribe directly to basemaps, eliminating the need for Apple to maintain a central basemap and pushing everything into the Overture ecosystem - at this point, Apple Maps would simply provide scaffolding for spatial appmakers.

Imagine seeing your Uber car, DoorDash delivery and your husband’s shared Lyft ride all converging on a friend’s house for a birthday party! Imagine having your Lime scooter automatically populating with your hotel address, or using Handoff to send data directly from desktop / web to your iOS device for easy navigation.

Now imagine this as context for Apple Intelligence - allowing Siri to answer a whole host of critical “who, when and where” questions based on the live information flow into the on-device spatial intelligence engine. Imagine landing in a new airport, trying to get to a tight connection - maybe your United App could provide turn-by-turn Apple Maps directions based on LOD which was being dynamically injected by the United airport map. Or asking a HomePod when the food was arriving while you’re trying to prepare for a dinner party. Or using your Apple Watch to order a car to the airport while you’re frantically packing for an international trip. The background context being injected into a central Map is effectively identical to the user’s mental context - key information about their life which is critical for answering their most important questions.

From a developer perspective, iOS would be able to provide gazetteer elements which synced across apps, GERS-focused advertising (“AdSpots” vs “AdWords”) for spatial apps to bid on to drive app downloads, and to push users into a more app-friendly environment where incentives are more aligned (vs Google where the goal is to own the end-to-end experience and take a large cut via referrals and ads). It could be a big way to get the industry on board with trying to shift users en masse off of Google Maps.

Meetings are the Dark Matter of Enterprise Cybernetics

We are going through is a transition period - from human reasoning to machine reasoning; given the previous revolution from human computing to artificial computing, I guess you could just say that the broad ~100 year arc is from human intelligence to artificial intelligence and just call it an AI revolution [1].

As such, I've been thinking a lot about how to use AI in the context of an existing business to streamline internal operations.

And while pieces of the business feel tractable based on applying AI to existing systems of record or operational processes, automating large swaths of a business feels too hard. Too much reasoning is illegible to machines - because it happens during meetings.

Meetings are the dark matter of the enterprise - the vast majority of "context" about what's happening at a business is transmitted verbally, and needs to be represented digitally. So a key part of transforming between the analog business to the digital business is about making meetings legible to machines.

That's why I think Granola.ai is going to be the Killer App™ for the next generation of enterprise operating systems - I'm only testing it out right now in a personal capacity (it's currently quite limited from a security perspective and not enterprise-ready), but it's the first application I've used that really feels like a step-change for personal productivity. Chat apps are nice for search, and I think they will continue to be useful. But they still feel like work - the experience for most LLM chat apps is still relatively similar to a better search engine (you swivel chair to it, do stuff, then swivel back). Granola is the first app I've used that's non-zero sum with my time; it inhabits the same time as I do, and makes that time more productive. I can't explain it exactly, but using Granola feels like - oh yeah, this is going to be ubiquitous in 5 years. Maybe it doesn't win the category (will be a battle), but this category is going to be the first non-chat Category of LLM-powered apps.

The problem is that in its current form, it's just a tool. It's going to be picked up ubiquitously and let people do better meetings - what it's NOT going to do is automate away meetings or massively accelerate operational productivity.

That's where the Ontology comes in - because right now, Granola is just a generic application. The opportunity is to use the Ontology to turn it into a Platform. Today, Granola has a basic templating system with generic out-of-the-box templates for specific meeting types. By applying a decision-centric data model to it, you could map meeting types to object types, and then include functions and actions in the UI where it currently provides some generic out-of-the-box actions (send email, list action items).

So now you've created a virtuous cycle - you use Granola to capture meetings, those meetings become data about the actual reasoning and decision-making in the enterprise, and then the meetings can be automated, orchestrated, agentified...ontologized.

Today, there's a feature in Granola that creates "action items" - imagine if each of these action items corresponded with an Ontology Action Type - with the right ontology, the Action Items from a partnership kickoff call could all be invokable Ontology Actions - one to create a Jira ticket for reviewing API docs, one for initiating a Legal ticket to get a partnership set up, one for scheduling a kickoff meeting, and one that automagically created a Slack channel. And they don't need to be actions yet - just having the Meeting <> Action Item mapping for all meetings of a given type lets you begin mining the plaintext action items to create semantic action types (and an Action Type <> Prompt Hint mapping so that future Action Items could be translated into invokable Ontology Actions via semantic search).

At this point, the UI would transform the text into a clickable bullet automatically orchestrating an entire system of action which could be kept in sync as more meetings occur and actions move through a Markov chain of semi-formalized state changes - "okay, you decided to transfer Phil to the Mobile team - let's kick off the process of talking to Phil, confirming the transfer, and then registering this."

To automate decisions, they need to be formalized and given that most decisions happen in meetings, this means that automating decision-making will require making the meeting context accessible to machines. Over time, this will enable us to progressively titrate decision-making authority from the human to the machine as humans shift from being decision-makers themselves to being the makers of decision-making machines.

[1] It's only tangentially related, but I do think that this Eric Schmidt talk was an interesting read about the steam to electricity transformation, which might be a good historical analogue to consider.

Planning as Statetime Manipulation

"All happy families are alike; each unhappy family is unhappy in its own way."

[1,1] [1,0] [0,1] [0,0]

Successful large technical systems represent a bounded region of a near-infinite space of possible configurations.

Unlike purely mechanical systems, which are defined purely by a limited set of physical dimensions, the dimensionality of LTSs is highly abstract, often consisting of intricate couplings between sub-systems and nuanced relationships between both the human and artificial agents who perform actions (who have agency) in the context of the broader system.

Making large-scale changes to these systems is the responsibility of the organizations which steward their evolution. Traditionally, this is done by conceptualizing a vision, developing a strategy to achieve that vision, breaking that strategy down into a plan, and then executing the plan.

The words we use to describe this activity, which is the core activity that all system developers (and therefore all modern leaders) must excel at, are anchored in skeuomorphism - they are loanwords from a prior era in which the largest systems (armies, cities, factories) were primarily physical entities.

In our cybernetic age, though, these words are mere analogies.

Modern system design is not just about manipulating space; it is about manipulating state - and that means that the most critical aspect of system design is figuring out how to represent a given system's state. What is the right way to conceptualize it? What set of words, diagrams and quantitative metrics should be used to describe the space? How to balance accuracy with simplicity? How to evaluate different representations?

Put another way, the first step of manipulating state is making a map.

Maps are an ancient tool for representing physical space; schematics and blueprints are a more modern tool for doing the same. The tool for representing state space hasn't really been developed yet - in most organizations, it's most commonly represented in Excel, often for the purpose of financial reporting and budgeting. But I think that the more appropriate tool is probably a variant of the ontology system we developed at Palantir, and the process of constructing an ontology is akin to cartography.

Mathematically, this map, or ontology, is effectively a manifold - a low dimensional representation of the high dimensional space. By definition, this description is lossy - the map is not the territory, all models are wrong etc. Pragmatically, though, some models are better than others - they can be manipulated more productively.

In this endeavor, it's fascinating how many words we use derive from this notion of a map. Vision is about describing a particular region of state-space in enough detail to communicate it broadly. Strategy is often described as "charting a course" to that future vision and avoiding barriers. A milestone, which literally derives from a physical stone placed along a one-dimensional road or path, represents a discrete position within state-space. A single team's roadmap may represents a vector connecting multiple milestones while the organization's roadmap is a zoomed out version representing all of the vectors actively being manipulated by the organization.

These analogies are helpful, but they miss the key point. Unlike a traditional map, which represents 2D projection of our 3D globe, our map is purely imagined. It does not exist in any absolute form. Of course, this is loosely true in cartography as well. The NYC subway map is designed to show subways and warps the region for aesthetic and practical purposes. But spatial cartography is always anchored in physical reality - the territory is at some philosophical level observable, while ontological cartography is a more unbounded exercise (see also Cartographic Grounds: Projecting the Landscape Imaginary for a whimsical overview of cartographic creativity).

To conclude, our modern software systems are highly dimensional; but our mechanisms for understanding complex systems have evolved based on our need to understand lower-dimensional, predominantly physical systems.  

This doesn't make software easier or harder to design and manage - it makes it different. Hardware is hard because doing it is hard; software is hard because defining it is hard. Software has eaten the world, but our ability to conceptualize and visualize these cybernetic human-computer symbiotes has lagged our ability to build them. At some visceral level, we do not understand what we have created. This is not uncommon; the dance between theory and practice is rarely linear.

But the conclusion is clear - in an era where theory has gone out of fashion in science, it's more critical than ever in business. The idea of a map only gets you to the starting line - creating the right map is the foundation of everything else in a modern business.

Anti-agglomeration Policies

 The entire developed world is facing an extremely acute housing crisis; as agglomeration effects drive more global talent into a smaller number of international supercities, local incentives bias towards increasing real estate values. These incentives are in direct conflict with a clear national and international interest in increasing the rate of technological development, as real estate extracts an increasing amount of value from the innovation created via agglomeration.

Historically, technology development was widely distributed - the unique geographic features of distinct regions and the high cost of transportation allowed individual cities to leverage their unique natural environment to facilitate the development of localized supply chains and innovation hubs. Toledo was a glass city, producing windows and windshields for Detroit; Pittsburgh's access to iron and coal made it a natural place where steel could be produced.

However, with the rise of container transportation, global air travel and widespread internet access, goods, people and information are now able to move across space much more quickly. As a result, the physical barriers to agglomeration have been largely eliminated - the natural result is a drive towards consolidation around a smaller number of winning cities.

This effect is particularly acute in America, a country accustomed to boomtowns and their inevitable busts. But it is not limited to America. Innovation-driven capitalism naturally optimizes for agglomeration as density creates the human connections which fuel the development of new ideas and creation of new products and services.

One obvious solution is to examine the spatial distribution of major government functions. The United States is a federal republic; our government does not need to be geographically localized in Washington. By intentionally picking new locations for critical government agencies, policymakers could redirect their employees to communities which need investment. Much like universities can create geographic gravity wells around college towns, moving large swaths of the federal government into states would create similar hubs - helping agglomeration revitalize cities across the country that have been left behind by our increasingly globalized logistics systems and airborne travel networks.

It should be easy to organize a dedicated lobbying effort to rally Congressional support to eliminate all executive presence in Washington, DC. The Department of Agriculture could move to Des Moines; HHS could move to Cleveland; the Department of Energy could move to Houston; the Department of the Interior could move to Bozeman. Put each agency close to a city which already has an affinity for that kind of work, but which has lost population over the past 50 years. Critically, this should be extremely popular within the legislative branch - with the exception of Virginia and Maryland, every state could be a winner of a meaningful part of the ever-expanding federal bureaucracy.

Cyclists, Drivers, Runners and Pedestrians

One challenge for urban designers is that cyclists exist in an awkward superposition state between pedestrians and drivers - bikes are larger than people but smaller than cars; cycling is faster than walking and slower than driving.

Across the road network, therefore, we've generally accepted that dedicated bike lanes are an appropriate compromise - they create dedicated space for bikers ranging from 10 mph to 25 mph at the cost of ~6 horizontal feet.

The rise of electric scooters and bikes, however, necessitate a similar segmentation on mixed-use trails - because electric assistance increases the median speed of a bike has increased from ~10-15 mph to 15-20mph, the gap between both walkers and runners and cyclists has become much larger. Especially on highly-trafficked trails, situations in which two cyclists at different speeds are trying to pass pedestrians is becoming increasingly common.

The solution is to split off a packed gravel running and walking trail from an asphalt-based cycling trail. It should be well-graded to support strollers and ideally isolated from the biking trail as much as possible - e.g. by putting a pedestrian trail on one side of a river and a bike trail on the other side.

Leaving Palantir

Most well-known technology companies achieved their initial success by creating a singular breakout product which dominated an emerging category. Each story is different, but the pattern is similar.

But even after twelve years, the uniqueness of Palantir’s story continues to fascinate me – the lack of explicit hierarchy inherent in the company’s design creates a superposition of possible company structures, each only accessible from a different vantage point. And the symbiosis between us and our customers makes untangling what we do from what they do especially challenging.

Studying Palantir has been something of a hobby of mine over this past decade, and over the past few months, as I’ve started to feel my time here coming to an end, I’ve tried my best to distill my reflections down into something transmissible.

In my time here, I’ve been right about many small things, but I was wrong about one big thing – programs, not products or platforms, are the defining concept at the heart of Palantir.

Shrink-wrapped SaaS products may be good businesses, but they are a bad way to shape the world predicated as they are on an end of history worldview which implies that zero-sum optimization and financial engineering, not positive-sum growth and technology development, will be the defining activity of our generation.

Cloud-based computing platforms are also good businesses, but their optimism is indeterminate in nature, with a teleology defined by their users; a platform is a tool, but it has no intrinsic purpose. Great platforms are always a consequence of successful products, and their value is always defined by the next generation of products built on top of them; they are supporting actors solidifying known patterns into a strong foundation, not prime movers pushing on the ceiling into the unknown.

Programs – living cybernetic systems consisting of data, logic, and action – are the things that Palantir has learned how to build, at extreme cost. Programs are n-of-one entities; instances which are members of a class, but with unique identities, designed to be different and therefore to differentiate. We have collectively built these programs as an output of a tightly integrated business model – our best BD work combines elements of strategy and technical consulting, and our best PD work intentionally blurs the line between traditional software development and a classic services approach.

Producing programs is what Palantir was designed (has evolved?) to do.

But perhaps our work here has an even higher purpose, bordering on the spiritual – to challenge those of us who pursue it, and prepare us for our next adventures. Why did PayPal and General Magic, not Google and Facebook, spawn Silicon Valley mafias? Where are the Snowflake and Databricks founder mafias? Why did my brothers and I work for ABL, Tesla and Palantir, not Boeing, Ford, and Microsoft?

Culture, not technology, may end up being the main thing these institutions end up contributing to the world; people may turn out to be their most enduring legacy.

Palantir has a very special culture – of agency in the face of bureaucracy, of engagement with the world, of curiosity about how things work coupled with an optimistic belief that the future can be better, in concrete and achievable ways, than the present. A community of pragmatists, programmers, and philosophers. It’s a quintessentially American culture that transcends American geography, a melting pot of ideas and people whose whole is greater than the sum of its parts.

We are, and have been for some time, in the early stages of going supernova – spreading this unique culture across the broader industry and society, and I’m excited to join the diaspora.

Since I joined Palantir twelve years ago, there’s never been a bad time to join, and there’s never been a good time to leave; my work here isn’t complete, partially because it isn’t the kind of thing that is completable. The beauty of building a mission-driven company is that even as each campaign ends, the movement itself regroups to focus on the work ahead.

And yet the time has come to leave - to peddle my wares on the open market of late-stage capitalism and sink my teeth into a new challenge. I'm stepping onto the well-trodden path from software to hardware, joining an exquisitely capital-intensive space company and hoping to help them build an integrated hardware + software product strategy, something I've always been fascinated by theoretically; but as they say - in theory, there's no difference between theory in practice, but in practice, there is!

Perhaps I can even bring an ontology into my new board room, and into our new operations centers - the most forward-deployed engineer.

VLR!