Extracting every ounce of learning from your actions is critical to solving hard problems. Every time you poke at the world is an opportunity to discover something new about the dynamics of a problem space.
Even for a small team, maximizing learning is hard. It requires discipline to routinely loop back to your previous endeavors to analyze what worked and what didn’t. It’s much easier to leave the past behind and blast forward to the next enticing plan.
For small teams, at least, it’s easier for contributors to remember the outcomes of their previous attempts and share insights. A startup’s ability to learn enables nimble pivots en route to the promised land.
Multi-team companies face larger obstacles in the learning process:
It takes energy for contributors to communicate their learnings widely.
Knowledge doesn’t easily translate across departments or cross-functional roles.
Knowledge walks out the door with attrition.
For new employees, the learning process starts from scratch.
Today, maximizing learning at scale is almost impossible which takes away some advantages that bigger companies should have:
Companies with longer histories should have accumulated more learning to make solving future problems easier.
Companies with more people should be able to create a network effect of knowledge transfer across teams.
The Double-Loop master plan is to remove the obstacles that prevent learning at scale. Here’s how we’ll do it.
The foundation of learning at scale is recording launches and results. Much of the data already exists in project management, deployment, code versioning, and analytics tools. Humans must add context such as strategies, goals, hypotheses, pictures, and results summaries.
Everyone in the company should be able to create, access, search the history of launches and results.
In realtime, every contributor should be able to follow the actions of other contributors that relate to their own work.
At the bare minimum, this can be accomplished by autogenerating high-level summaries, distributed by Slack or email, based on the record of launches and results.
But true learning at scales requires granular notifications. Teams should be able to subscribe to targeted facets of the launch record. For example, for a particular product change, customer support might need to know the details the UI while the sales team is more interested in the impact on the overall value proposition. Similarly, an engineer working on SEO should be able to see what other teams have done in the domain, what’s worked, not worked, etc..
While record keeping and communication provide the building blocks of learning at scale, there is potential for software to play a new role to amplify memory and learning. Here are a few ideas.
Software can automatically generate a timeline of product launches based on deployments and project management software. Given the trend towards high-frequency, small deployments. Tools are needed to separate the signal from the noise.
Based on the above, algorithms can guide team members to communicate or analyze the most important launches and retrospectively analyze results. Imagine a feed of product changes, consumed by data scientists, ranked by their propensity to impact business metrics.
Software can learn which launches across a big company you care about to create real-time feeds of knowledge cross-pollination.
Based on defined key results, software could (A) automatically classify the success level of product changes based on app analytics and (B) train a system to predict success likelihood in advanced of engineering commitment based on the structure of plans in project management tools.
I believe we’ve only scratched the surface of systematically cultivating learning in the innovation process.
Innovation requires a combination of work and meta-work. By “work” I mean doing things that tangibly impact your product; designing, coding, marketing, etc.. “Meta-work,” in contrast, involves improving how you’re working. This often looks like meetings to synchronize the activities between teams or applying frameworks like OKRs or lean startup.
Over-emphasizing work, while neglecting meta-work, turns you into a feature factory where you’re producing in high quantity, but the stuff you’re making is of questionable value.
Conversely, spending too much time on meta-work often entails a day full of meetings and bureaucratic processes when no work is actually done. Large companies have no choice but to spend lots of time on meta-work since they need to coordinate the work of many specialized contributors. They must maintain safeguards to prevent the loss of market share already won. This in part explains why big companies are so prone to get disrupted — they don’t have time to actually work.
Growth stage companies, however, are in the predicament of having to consciously decide how to balance work and meta-work.
Relentlessly working without pausing to reflect and communicate might succeed with a small team of founders, but it doesn’t scale to new employees and multi-team orgs. Meta-work is required to prioritize the work necessary for solving future problems that can’t be felt viscerally at the present moment.
But speed matters too. If you don’t “move fast and break things,” it’s harder to sneak up and capture a new market or catch an incumbent player flat-footed.
The tension between work and meta-work, in part, is why we’re seeing a rise of tools focused on meta-work. While there are tons of tools for getting work done, project management tools, dev tools, design tools, etc., there are fewer tools that change how we work. Now we’re seeing a growing market of product management tools that help teams communicate roadmaps or track their progress against OKRs. My project, Double-Loop, helps teams learn and communicate by recording a timeline of hypotheses and results. This is faster than the meta-work task of sending a product launch email or slides.
These tools remove the tension between work and meta-work by making the meta-work more efficient. If you can spend less time getting the benefits of meta-work, you have more time for the actual work. You can optimize for speed and value.
Stand-up bots, like geekbot, are the most literal version of this. Instead of spending the time on synchronous meetings to synchronize, you can synchronize asynchronously; fewer people have to break their flow to attend an in-person meeting. Slack is powerful because it’s a tool for completing the necessities of work with meta-work automation mixed in.
Automating meta-work is a little bit like automating the role of the manager. It’s the manager’s job to preserve team harmony, create accountability, and set strategy. Meta-work tools allow teams to do this bottom-up.
However, elements of meta-work resist automation. As the head of product for a growth stage company, just because I ask my team to use meta-work tools doesn’t mean they actually will. It’s hard to prioritize meta-work tools that can feel like distractions from pressing tasks.
I discovered that meta-work tools, ironically, require meetings to drive usage. I can ask my team to use our new OKR tool, Gtmhub, but no one would use it if we didn’t have a meeting to review the data we enter into the tool. Similarly, my team wasn’t using Double-Loop until we structured meetings around looking at Double-Loop to look at product launches and results.
The designers of meta-work tools should prioritize making screens that are specifically suited to be projected on the wall during meetings. Meetings are the lifeblood of these tools. It’s not the job of meta-work tools to completely kill meetings. However, it is their job to make meetings less frequent and more efficient.
The best meta-work tools minimize duplicate data entry. Meetings are especially costly if you need to spend an hour updating a spreadsheet before the meeting. As much as possible, the data in meta-work tools should be aggregated from the actual work tools.
In Double-Loop, we’re building a timeline of product launches that can be understood outside of the tech team, yet the building blocks of each launch event come from Jira. The team still needs to add new information for context, but I believe the Jira bootstrapping will be key to driving adoption.
Teams, in general, are getting more thoughtful about designing their toolchains, and companies like Segment, Zapier, and Unito make this easier. To take this one step further, I believe companies need to think beyond tools chains and design meeting+tool-chains optimized release the tension between work and meta-work.
Making the transition from hands-on product management to leading a PM team can be disorienting. On the one hand, if you get overly involved with projects, you’ll stifle the creativity of the team. And it’s unsustainable to be on top of every detail, especially with products that require heavy domain knowledge. On the other hand, if you’re too hands-off, you risk allowing product changes that don’t meet your standards or big picture business objectives.
More than once I’ve seen product leaders, unable to grasp their role, get arbitrarily involved in design decisions, seemingly out of a need to put their own “stamp” on the product. Behavior like this demoralizes the team and creates a low-value hoop in the product delivery cycle.
But how far can you step back? The key is minimum viable control (MVC). The essence of MVC is managing the structure of your team’s process without controlling the substance of launches.
The Elements of MVC
I. A sound learning process
As a manager of PMs, as much as possible, I stay away from questioning the ideas and designs that come through the development cycle. However, I need to feel confident that each change is being sufficiently tested, validated, and evaluated post-launch.
A sound learning process requires teaching your PMs to have a scientific and paranoid mindset. PMs should relentlessly pick apart and test each assumption underpinning product directions. They should be paranoid in imagining everything that might go wrong with each product change and take necessary measures (e.g., through A/B testing) to avoid tanking business metrics.
I created Double-Loop, in part, so teams can quantify the rigor of their learning process. With Double-Loop we can measure and continually improve how many of our launches have clear hypotheses and results.
II: Cross-functional integration
As I discuss in The Product Management Triangle, near to the essence of product management is synthesizing the activities of the diverse set contributors spanning engineering, design, marketing, sales, and customer support. As a product leader, you must ensure that your team is incorporating inputs from across the organization in their ideation and prioritization process.
Conversely, your PMs should treat internal communication with the same care as external communication. When sharing a product update or strategy, PMs must tailor their medium and message to the unique psychology of their cross-functional counterparts. For example, after a launch, your team may need to communicate different information to marketing than they do to customer support.
As the product leader, you must create the bidirectional pipes of communication between all the people who touch the product. Your job is not to oversee what content goes through the pipes. Instead, it’s to make sure that the right pipes exist, unobstructed
III. Top-down integration
You have better visibility than your PMs into the big picture of the business. You’re more exposed to the workings of the board, dynamics of the executive team, and financial milestones. In contrast, your PMs should know more than you do about the functional workings of their product area and the nuances of their domains.
Your job is to ensure that the work of your team reflects the big picture reality without dictating what your PMs build to achieve the goals. Objectives and key results (OKRs), for example, serve as a nice non-prescriptive interface with overarching business direction. The best-formed OKRs provide stable criteria for what success looks like while not constraining tactics. Without something like OKRs in place, your team will feel like they’re floating without a foundation to stand on.
Top-down integration must go both ways. While your team’s work must reflect objectives from up high, the learning of your team should influence how the executive team thinks. Your team must abstract and communicate the results of their launches in a format that is clear from way up in the sky. Otherwise, antiquated strategies will keep raining down.
Maintaining a sound learning process, cross-functional integration, and top-down integration is a full-time job that requires surprisingly little knowledge of the details of your product. In a sense, the product leader provides value as an outside observer who can objectively monitor how their team is working without micromanaging what they’re doing.
By staying at the level of MVC, you can achieve the best of many worlds. Your team will be unleashed to solve problems and seize opportunities as they see fit. Embedding your PMs in a rich network of horizontal and vertical information flow will fill their brains with the context to make wise decisions. And the scientific process of testing and experimentation will make their failures safe, both financially and psychologically.
The tools we use to make products shape how we think.
If you look at the saturated market of product development tools, the majority revolve around the planning and execution leading up to a launch. The apps cross a spectrum covering engineering workflow (e.g., GitHub), project management (e.g., Jira, Pivotal Tracker), cross-functional handoff (e.g., InVision), and big picture road mapping (e.g., Aha!, ProdPad).
In project management tools specifically, there’s great satisfaction in finishing projects. Marking things “done” provides gratification similar to removing no longer needed items from a cluttered room. With a feeling of “Mission Accomplished!”, you can now move on to the next hot priority that fits your most up-to-date thinking.
The problem is that when you launch a product update, your journey is far from complete. Greg Davis from Intercom puts it well:
As your launch day comes to a close, it’s natural to pat each other on the back and wipe your hands clean as you head home. The hard part’s over. You launched.
The reality is you aren’t done…
To extract maximum value from a launch, many opportunities come post-launch:
Customer communication. A feature has little impact if users are unaware. There’s a diverse playbook of product marketing tools for notifying users ranging from in-line tours to email newsletters.
Internal communication. Stakeholders need to understand the new capability. Internal launch communications is its own art form requiring messages tailored appropriately for each audience. Your sales team has different information needs than your customer support team.
Measuring engagement and feedback. Through both quantitative analytics and qualitative feedback, you can assess the success of the launch and find inspiration for new opportunities.
Collective learning. After you launch something and measure its impact, you can evaluate the results relative to the original hypothesis to eliminate false assumptions. It’s powerful when your whole team moves through the learning process together.
Process retrospectives. Projects can go awry in a variety of directions. Post-mortems help teams untangle the factors that led to scope creep or creative tensions. Keeping a record of launches can help you optimize how you’re breaking large projects into bite-sized iterations.
Ongoing iteration. When looking back at my career, my greatest regrets are times when we didn’t follow through on our ideas after the first launch. It’s easy to bail on a direction if the world doesn’t take notice after the first launch or if company strategy shifts. But most big ideas require grinding out iteration after iteration to achieve impact.
Our execution-focused tools, on their own, insidiously cultivate a “launch and forget” mindset. After we launch something, it should not disappear from view. Instead, launches should linger around. We should be guided by our tools to follow up, create tailored messaging for each audience, and extract maximum learning value from external feedback and reflection.
Filling the post-launch tools gap is the vision for my project, Double-Loop. My mission is to provide a powerful place to keep a project after it goes live.
Double-Loop starts where the execution tools leave off. The Double-Loop Slack bot is automatically triggered when you deploy code. It prompts the team to record launch hypotheses and it reminds you to follow up on the results when time elapses after launches. Stakeholders stay in the loop through the automatically compiled launch emails.
This is just the beginning. I’m determined to demonstrate the lucrative whitespace in the post-launch phase of product development.
Marshall McLuhan, with his famous quote, “the medium is the message,” makes the point that the content of a message is less significant than the medium used.
McLuhan’s principle has profound implications for how to run a company. But first, let’s delve into the concept at play.
To illustrate, consider two parallel societies. Both societies have the same laws. In one of the societies, the laws are only spoken by the leader and shared verbally from citizen to citizen. In the other society, the leader speaks the laws but they’re also inscribed on walls and on paper.
In both societies, the content of the laws are the same but the media Is different. One uses speech and the other uses writing.
The difference in media has a profound impact. Through his research into societies with spoken versus written laws, the anthropologist Jean-Pierre Vernant concluded that the “[s]etting [laws] down [in writing] not only ensured their permanence and stability; it also removed them from the private authority of the [ruler], whose function was to ‘speak’ the law” (as quoted by Levi Bryant in Onto-Cartography).
The sound waves of speech can only travel so far. Its content transforms from one listener to the next and fades away in memory. Without inscription, a spoken law is a commandment that the leader can easily alter with minimal accountability. The leader issuing laws is the lone authority on what the law actually is.
When a law is written down, in contrast, it takes on an existence independent of the leader who uttered it. This leader can still change the law, but they must reconcile modifications with the law in its previous form. Bryant explains:
… insofar as the law has been inscribed or written down, it takes on an objective value that rescues it from the whim of the ruler. In being inscribed, the law becomes a thing or machine in its own right rather than a command of the ruler. To be sure, the law written down issued from the ruler, but in its inscription on parchment or the wall of a temple, it now takes on an alien existence, an independent existence, that the ruler himself must contend with. Today he might be inclined to enact a different law, but because his “speech” lingers in the form of the written document, the ruler now finds that he must mesh what he said last year with what he wishes to decree today. Indeed, not only must he contend with what he said last year, but he must also contend with what previous rulers inscribed.
Written laws can be copied and proliferated without changes to the content. While interpretations of the law differ for individuals, the words themselves aren’t subject to the game of telephone.
Consequently, societies with written laws can scale across larger geographical areas than societies with only spoken law. It is not the laws themselves that facilitate the scale but the medium in which the laws are communicated. This is what McLuhan means by “the medium is the message.”
You can apply McLuhan’s principle to corporate communication. While it’s tempting for leaders to focus foremost on creating the right strategy, the mediums they use to communicate strategy are more important than the strategies themselves.
Strategic direction must constantly shift with signal from the real world. A leader’s pivots risk losing the confidence of the team. By writing down and sharing the rationale behind a shift, a leader makes the new strategy easily transferable and projects sound reasoning throughout the company. When strategy changes are only discussed verbally, in contrast, the intent can be lost in translation leaving contributors feeling jerked around by the whims of management.
Just as lack of written law inhibits a society’s scale, lack of written strategy stifles startup growth.
More companies, on the growth path, are following Google and The Gates Foundation in using OKRs (Objectives and Key Results). OKRs are a framework where teams and individuals define their goals and measurable criteria indicating progress. OKRs are captured in spreadsheets or tools to be shared openly with the company at large.
The rise of the OKR model demonstrates that more companies recognize that the medium is the message. Brilliant strategies raining down from the executive team are insufficient for success as teams scale. Instead, a culture of open, stable, and written strategy is required to navigate an evolving landscape.
My project, Double-Loop, is another application of McLuhan’s principle. Double-Loop creates the expectation that product developers explain the hypotheses of their launches and loop back to examine the results relative to the original goal. The result is a culture of scientific product development. Double-Loop reflects that the mediums you use to communicate your product ideas are more important than the ideas themselves.
Mediums, in McLuhan’s sense, go beyond what you might traditionally think of as a medium. Levi Bryant explains:
We can see just how broadly McLuhan expands the concept of media, giving it a much deeper ontological significance than it is often taken to have. Often when we think of media we immediately think of things such as newspapers, television, music, etc. While these are indeed examples of media, McLuhan expands the notion to include everything from forks to seeing eye dogs. McLuhan thus recovers the Latinate sense of media as medius, denoting “intermediary.” A medium is an intermediary that relates one thing to another. Thus, for McLuhan, a medium does not so much refer to a particular medium of communication such as speech, sign-language, radio, television, writing, or smoke-signals – though all of these things are included in his theory of media – but rather places emphasis on both the materiality of media and the specific nature of that materiality, as well as the manner in which these media extend and amplify our sense-organs.
Bryant goes even further than McLuhan in arguing that machines can be mediums for other non-humans machines. In this sense, for example, your analytics systems are mediums for your website.
McLuhan and Bryant’s broad understanding of media is a valuable lens for understanding how a company’s systems and tools profoundly impact long-term success. What your company is doing at one moment is less important than the mediums its selected to structure the behavior of multiple generations of employees.
For a few years now, I’ve been obsessed with critiquing the notion that the great creators bend reality to their will. Some think that Steve Jobs, ignoring naysayers and customer feedback, changed the world to fit his vision. But in actuality, no leader is powerful enough to manipulate materials and people as they want. It’s always a negotiation. And to take things a step further, our creations design us as we design them.
The idea that designers can frictionlessly bend materials to their whims connects with the ancient philosophical tradition of hylomorphism. Rooted in the thinking of Aristotle, many Western philosophers describe matter as existing independent of form. In this model, it requires a “designer,” god or human, to add a meaningful structure. Levi Bryant explains:
The term “hylomorphism” comes from the Greek hyle signifying “matter” and morphe denoting “form.” Under this model of fabrication, the artisan first has a sort of blueprint of what he wants to produce in his mind (the form), and then imposes that model on matter giving it form. I first have a mental model of the knife I wish to produce in my mind and then set about fashioning the materials of the world about me into that form.
Yet when we look more closely at the actual activity of fabricating a work of art, tool, or technology, we see that something very different takes place. To be sure, the artist has some sort of intention to produce something like shelter from the elements, and this intention can involve a more or less elaborated model as in the case of an architect’s blueprint, but this is where the similarities to the hylomorphic model end. The problem with hylomorphic models of how artifacts are produced is that they forget both the time of production and engagement with the materials of the world. What attentiveness to the time of production and engagement with matter reveals is that the production of any artifact is much closer to a negotiation than the simple imposition of a form upon a passive matter. And as is the case with all negotiations, the final outcome or product of the negotiation cannot be said to be the result of a pre-existent and well-defined plan.
Bryant contends that all things, ranging from computers to mud, have their own structure that we must contend with. It’s easy to concede the unpredictability of inserting a new invention or product into complex social systems. But it’s tempting to think that, in confined domains, artists can have their way with the materials in front of them. However, Bryant argues that all materials have their own structure that they impose on us. He writes:
Take the sculptor working with marble. They might begin with a vague idea of what they want the marble to become and even select specific pieces of marble to execute this local manifestation, yet as they begin to work the marble, encountering its grain and veins, they’ll talk about how the marble “wants” to become something else.
Movements in software development such as agile and lean startup reflect our society’s letting go of hylomorphic design thinking. These methodologies involve incrementally poking the world to see what happens, before getting too far ahead of ourselves.
But we still have a long way to go towards appreciating the hidden powers of our own creations, and the implications are severe.
Take the invention of a clock. Did the creators of the first clocks intend to create a world where functioning socially and economically demands synchronizing with others according to a fixed absolute schedule?
The creators of internet-driven social networks like Twitter and Facebook had idealistic visions of democratizing information and “making the world more connected.” They did not know that their invention was going to, in turn, create Trump and tribalism.
The first car manufacturers did not know that they were creating a culture where humans, even with knowledge of climate change, would be so dependent on cars that they can’t stop polluting.
In web development, when we’re unsure of how the world will respond to a design change, we can A/B test it and then remove the change based on the measured effect. But with other design “innovations,” it’s not so easy to put the genie back in the bottle. The largest human creative endeavors are like going to war in Iraq with no exit strategy.
We need new frameworks to anticipate the second-, third-, and fourth-order impacts of the things we create.
Want to get better at anticipating the impact of your creations? Try Double-Loop. It’s not going to prevent climate change, but it’s a start.