experiments – Faster Than 20

From Go-Go-Go to Going Slow: Lessons from My Own Painful, but Ultimately Successful Journey

I discovered weights freshman year in college, and as a weak and skinny kid, I found them to be a revelation. One of my friends had played football in high school, and he and I would work out together often. He worked out hard — at least five days a week — and I liked his routine, so I adopted something similar.

I rapidly gained strength, but I also peaked quickly. Naturally, I tried working out more, but it didn’t seem to help. I maintained more or less the same routine throughout my sophomore year without any significant gains, which I found frustrating and demotivating. It led to long stretches where I would stop working out altogether.

Junior year, I restarted my workouts, and again saw little progress. My next door neighbor that year turned out to be a serious bodybuilder. One day, I asked him for advice. I started by telling him how much I worked out, and he cut me off. “You’re working out too much,” he said. “Your body needs rest to recover and build muscle. Try doing 45 minutes three days a week.”

“Are you sure?” I asked incredulously. Everything he said made sense except for his specific recommendation, which amounted to half as much as I had been doing for years.

“Try it,” he insisted.

I did. After just a few weeks, I saw marked improvement for the first time in over a year. I was floored. It turned out that, in order to get stronger, I just had to do less.

The Power of the Pause

I am often approached by groups with go-go-go cultures asking how to:

Build trust and morale
Work together more strategically
Learn via experimentation
Align their structures and behaviors with their desired values and culture (often, these days, with a focus on equity)

When I hear more about their current situations, I often find myself channeling my bodybuilder friend. You can't do any of these things if you aren't regularly slowing down to pause, to:

Reflect together
Listen deeply to each other
Have hard conversations with each other
Make adjustments
Rest and recover

If you're not committed to developing your muscles around pausing, you not only will not succeed at any of the above, you may even hurt yourselves.

When my friend told me to try working out less, I was more than happy to try, even though I was skeptical. When I tell other collaboration practitioners they need to practice pausing, I get the same skepticism, but none of the joy. There are reasons why they are go-go-go, and trying to shift those habits and mindsets is not only very hard, it can be downright anxiety-inducing.

I know this all too well. For the first dozen years of my career as a collaboration practitioner, I was the worst perpetuator of this go-go-go mindset. It took me six years of consistent effort and constant failure along with a health scare to learn how to slow down. It's made me a better person and a better practitioner, and it's also made me tremendously empathetic to others who are suffering from similar afflictions.

I blogged a lot about my journey as I was going through it, both on my personal blog and on this one. I also shared a tool — my Self-Care Dashboard — that ended up being enormously useful. But I never bothered writing about how I eventually turned a corner, how I've been able to sustain this balance, and the impact it's had on my life and my work. I'd like to correct that here, both to complete the record and also hopefully to offer actionable encouragement to other practitioners dealing with this challenge.

Developing Bad Habits

The first thing to know about my go-go-go ways is that I wasn't always this way. I was capable of tremendous focus and endurance, especially when it came to things I was passionate about, but I also valued my chill-out time. My childhood was spacious and wonderful, full of time to think, explore, and zone out. My first few years after college were similar, which enabled me to realize my passion for collaboration as well as to develop some initial practices.

My lifestyle started to change for the worse when I cofounded my first collaboration consultancy in 2002. I was extremely fortunate to have a mentor who taught and encouraged me and a cofounder with whom I could play and learn. But I was also in my 20s with no formal experience and a job description that I had more or less made up. Moreover, we were trying to make do in a down economy.

Needless to say, business was not good. I went into debt and barely scratched a living for several years before things slowly started turning around. It was stressful and unhealthy, and even though I was barely making any money, I found myself working all the time. I was also young and single, and I suffered a bit from the Silicon Valley mindset that idolatrized struggle before success. I thought I was simply paying my dues, like any good entrepreneur or changemaker, and if business ever got better, I promised myself that I would return to my more balanced ways. Unfortunately, I was not precise with myself about what "business getting better" looked like, and my difficult habits and scarcity mindset continued to perpetuate themselves.

In 2009, three things converged, causing me to finally reconsider my ways. First, I experienced the painful end of a long relationship. Second, I was massively burned out. Third, my then four-year old nephew, whom I adored (and still adore), came to visit San Francisco for the first time ever, but I was so busy that week, I barely spent any time with him. I was extremely upset about this, and it caused me to reflect deeply on what I was doing and how I could change.

First Steps

One of the first and best things I did was to hire a coach. She helped me to articulate a clear vision of what a balanced life looked like for me. Put simply, all of the best, most balanced times of my life had three things in common: basketball, books, and lots of time with family and friends. Imagining a life replete with these three things made me feel light and happy, and they became my personal North Star.

The next step was to understand what, professionally, was preventing me from having these things. Two answers quickly came to mind: Bad habits perpetuated by a fear of not making enough money and a constant feeling of isolation. I was lucky to have community, but what I needed were colleagues. If I could make more money, and if I felt like I was part of a team, I thought I might take a break every once in a while, which would hopefully and eventually lead to me to my North Star.

I still had more questions than answers, but I made two concrete changes as a result of my coaching sessions. First, I raised my rates for the first time in seven years. More money, I reasoned, would give me more space. I considered myself to be one of the top people in the field, and I had known for years that my rate was not commensurate with others, but my deeply engrained fear of not getting enough work had prevented me from raising them earlier.

Second, I decided that I would never take on another complex systems change project without an equal partner. This decision both thrilled and terrified me. On the one hand, I craved partnership. On the other hand, it meant that I wasn’t just raising my rates, I was essentially raising and doubling them. Would I ever get any work again? I believed, in my head, that I would. I had seen others do it, so I knew it was possible, but I was still really scared.

Fortunately, I was able to test these changes almost immediately. The CIO of a Fortune 100 company approached me about some possible work. He fit the profile of a lot of past clients, in that he had tried working with a few traditional (and very expensive) management consulting firms, he had been dissatisfied with their results, and he was looking for something more outside-the-box. He was already talking to some prestigious design firms, when a colleague mentioned me as a possible candidate.

I felt excited about the possibility of working with him, I knew that budget was not going to be an issue for him the way it might be for a smaller organization, and I already had someone in mind with whom I wanted to partner. I thought my chances of getting the work was low, which emboldened me to really go for it — to put together a team of folks with whom I really wanted to work and to propose what I felt would be the ideal project without constraining myself. When I put together my budget, I could feel my palms sweating and my heart beating. I had managed projects with large budgets before, but I had never before written a proposal for that large a sum of money.

Somehow, we got the work! He and his team turned out to be dream clients, and a few members of the team I had pulled together became the core of the consulting firm I cofounded one year later.

After we secured a verbal agreement with our client, we had to work through some bureaucracy. I had been planning a vacation a few months in advance, but as the date approached, we had still not agreed on an actual contract, and I felt old anxieties cropping up. I strongly considered canceling my vacation, but my partner insisted that she had everything under control and that I should not only go, but fully disconnect while I was gone. I listened to her, and everything worked out. The structures I had created and the people with whom I had surrounded myself liberated me to take my vacation, my first in eight years.

Falling Over and Over Again

I felt relaxed for the first time in my career, and it was showing in the work. I was excited about new opportunities, and I loved everyone with whom I was working. More importantly, my life was feeling more spacious. I thought I had turned a corner. Unfortunately, I had confused taking a few steps with walking. I still had a lot of inner and outer work to do, and — as it turned out — I was going to have to fall a lot more along the way.

Things started getting out of balance again when a few of us decided to formalize our partnership. We spent a lot of time thinking about what we wanted to build together and how. In addition to being great consultants and building a great company that modeled our values, I wanted to explore ways of working outside of consulting, as I was feeling like we were reaching the ceiling of the impact that consulting could have. But, in theory at least, I was happy for that to be a stage two project, focusing our initial energies on building a great consultancy.

Over the course of several months, we converged on three priorities for our first year, the third of which was, “Space for Renewal, Learning, and Play.” Everyone was fully committed and aligned around this goal, and we did some things well. We did a good job of protecting other people’s time. We instituted practices such as starting all of our meetings with checkins, a tiny, but much needed pause that enabled us to breathe and be human together on a regular basis. We started tracking our time, which we all hated, but which gave us real data to see how we were doing collectively and to make adjustments accordingly. I designated my Wednesdays as "play days," which at minimum meant no meetings and at maximum meant open time to read, experiment, or simply take a break.

Unfortunately, I was still falling back on a lot of terrible habits. Running a company is stressful. A lot of people are depending on you to bring in revenue and to create a healthy, thriving work environment. I also felt urgency to do more. On top of the day-to-day challenges of building a successful consultancy, I was anxious to at least start exploring models outside of consulting, even though we had agreed not to prioritize this that first year.

All of this pressure — some real, some self-imposed — kickstarted my superhero complex. It was important to me that everyone else on the team had balance in their lives, but I believed that I could make do without, at least temporarily. I was motivated, I was confident in my endurance, and I felt it would move all of us forward without harming anyone. Besides, it would only be for a little while. Once we got over the hump, I could focus on restoring balance for myself.

Of course, it didn't work out this way. As a leader, I was not modeling the behaviors I was professing to prioritize. Everyone noticed this, everyone felt stressed by this, and — fortunately for me — everyone called me out on this. Moreover, intense stress and not enough rest was making me a bad teammate. It also was impacting my health. I thought I felt okay, but I discovered at a regular checkup that my blood pressure was alarmingly high.

This was the ultimate wakeup call for me. I needed to prioritize balance immediately. My life literally depended on it.

Turning the Corner

Shortly after founding my company, I started tracking a set of self-care practices every week on a dashboard that all of my peers could see. It was stark to see how often I neglected all of my self-care practices, including the low-hanging fruit, such as going for a walk. The simple act of tracking helped me make sure I was always doing some form of self-care, which was an important start.

However, it also made me see that "some" self-care was not going to be enough. This forced me to explore more deeply why I wasn’t able to make time to take better care of myself. I realized that working made me feel powerful and in control. When I felt like things were going poorly in my personal life, I defaulted to working as a way of feeling better about myself. I needed to confront these patterns head-on. I also adopted some simple tactics that helped. In particular, I took up photography as a hobby, which served as a much-needed creative outlet, helped reconnect me to my community, and unexpectedly had a profound impact professionally.

All of this core work turned out to be critical for me to implement the simplest and hardest solution of them all. In order to work less, I needed to stop doing something. That meant taking something off my list, which I had never managed to do.

I decided to leave the company I had co-founded, which — to this day — remains the hardest professional decision I have ever made. Leaving helped a lot. I was able to maintain a modicum of balance for the next few years, but I noticed that I easily fell back into old habits. I re-focused on working less, even declaring my intentions here on this blog. Repetition helped, and I was finally getting the hang of being real with myself and taking things off my plate. But, as it turned out, I needed to do one more thing before I truly turned a corner. I needed to stop checking my email so often.

Turning off my work email before dinner and on weekends had long been on my list of self-care practices. They were the easiest to do, and yet, after three years of tracking, they were the things I practiced the least. All of my hard work finally made it possible for me to do these “simple” things, and my self-care scores soared as a result. More importantly, it felt good. It turns out my email behavior was a good leading indicator of how much balance I had and also a keystone habit that unlocked other important practices.

In September 2016, I went to my self-care dashboard — as I had been doing every week for over four years — and decided that I didn't need to track anymore. After six long years of failing over and over again, I felt like I had finally achieved the balance I was seeking.

The following year, my nephew and his little brother came to visit me in San Francisco for the first time since that crazy week in 2009. I cleared my schedule so that I could maximize my time with them. We played basketball every morning, we went on long walks, and we ate delicious food. As I drove them to the airport at the end of their trip, I started to cry, not just because they were leaving, but because I remembered what my life and work was like the first time my nephew had visited eight years earlier, and I felt grateful for how my life had changed since.

Takeaways

It's been five years since I stopped tracking my self-care practices, and I've maintained this balance since. Not only am I as happy, healthy, and fulfilled as I've ever been, I am a significantly better collaboration practitioner than I ever was. Just as my bodybuilder friend explained with weightlifting, to get better at my work, I “simply” had to learn how to do less.

I share these stories not because I've landed on some magic formula for achieving self-care and work-life balance. Everyone's story and circumstances are unique, and I don't want to pretend that what worked for me will work for everyone. I share these stories, because I want folks to know that self-care is really, really hard. If you don’t already have work-life balance, there are likely very real, very hard reasons for this — both internal and external.

Achieving balance requires hard work, experimentation, and tons of support, and — if your experiences are anything like mine — you will fail over and over again. Even if you manage to achieve balance, you will always have to work to maintain it. You will constantly face obstacles, and old habits and mindsets will continue to rear their ugly heads. At the same time, achieving balance will also strengthen your faith in the importance and power of going slow, which will serve as motivation for you to continue your practices.

This matters, especially for collaboration practitioners trying to improve the performance of their groups. Working with urgency is not the same as working urgently. In order to be agile and impactful, in order to learn as you go, in order to do values-aligned work, pausing regularly isn't nice, it's necessary. One of the most powerful acts of leadership — regardless of your job title — is to model this. It's hard, but it may end up being the most important and impactful work that you do.

Podcast: Play in new window | Download

Subscribe: RSS

The Secret to High-Performance Collaboration: Slowing Down

At my previous consultancy, we used to spend a considerable amount of time debriefing every engagement, big or small. We were meticulous in our analysis, nitpicking every detail.

Over time, I started noticing a few patterns. First, I realized that our debriefs were largely ineffective, because we weren’t taking the time to integrate what we learned. We needed to be reviewing past debriefs before new engagements in order to remind ourselves of what we had learned and in order to hold ourselves accountable to improvement. Without that additional space, our debriefs were essentially exercises in self-criticism and generating lists, two skills we didn’t need to be practicing.

Second, a few things began to jump out at me as I reviewed our long list of things we could have done better. Almost without fail, when we had “bad” engagements, someone had slept poorly the night before. Or someone had been working while sick. Or someone had forgotten to eat breakfast that morning. (Forgetting to eat was my personal bane.)

We had spent hours and hours and hours debriefing, and this is what we learned:

When we didn’t take care of ourselves, our performance suffered.
When we didn’t take time to remind ourselves of past lessons, we repeated the same mistakes.

One of my mentors, Gail Taylor, is always encouraging me to seek the simplicity embedded in complexity. What I’ve realized over the years is that, when I find it, I often dismiss it. It seems too obvious. There has to be something else.

Slowly, but surely, I’m breaking this habit, and I’m starting to see more clearly as a result. Which brings me to my biggest insight over the past several months.

The best thing we can do to improve collaborative effectiveness is to slow down.

This has been coming up for me over and over again with all of my recent projects and experiments.

I’m currently doing an experiment with the Code for America incubator in trying to help new companies establish good collaborative habits right from the start. PostCode (the company with which I’m working) is working at a startup pace, and they’ve had inevitable challenges as they move through their storming phase.

Fortunately, when problems crop up, they deal with them quickly. In those situations, they’ve often reached out to me about possible toolkits to help them navigate their challenges. My answer has been consistent: Slow down. They haven’t found the time (beyond the work we’ve done with them, which has been too constrained) for critical conversations about organizational strategy, culture, and group dynamics. It’s understandable. They’re under a tremendous amount of pressure, and in those situations, conversations about strategy and group dynamics can feel like a nice-to-have, not a need-to-have.

Last week, I facilitated a practitioners workshop for the Garfield Foundation on collaborative networks. For one of the Open Space sessions, I led a group through the power workout I developed for Changemaker Bootcamp. We had a wonderful, nuanced conversation about power dynamics, and several people asked, “How do we make sure we have more of these conversations with our constituencies?” My response: “The first step is making the time.”

Telling others (or even yourself) to slow down is easy. Actually doing it is hard. We move fast because of external pressures, mindsets, habits, cultural norms, and so forth. We have little control over most of these things, and what little we can control is incredibly hard to change. But there are tricks that I’ve found helpful over the years.

Experiments. Changing habits is hard, and you will likely fail many times. Approaching the challenge of slowing down as a series of experiments helps. As I wrote previously, one of the keys to a successful experiment is to hold yourself accountable to the results. Failure is okay as long as you’re committed to the intention and are actively incorporating what you learn into new experiments.

If you are truly committed to slowing down, write it down. Write, “Slow down,” on a piece of paper, and put it up where you will see it. Actively devise and track experiments that will help you do this, so that you can monitor your progress and see what’s working and what’s not.

For important conversations — whether it be organizational strategy or a project debrief — schedule those in advance as part of your project plan, and track whether or not they actually happen. If they don’t happen, take the time to examine why, and devise another experiment. The beauty of applying an experimental framework is that simply the process of devising and tracking an experiment is an act of slowing down.

Checkins. My former business partner, Kristin Cobble, recently wrote a wonderful piece on the power of checkins. She describes them mostly in the context of meetings as a way to invite participation and to garner a sense of the collective whole. What I’ve learned over the past several months is that they also serve a much simpler, deeper function. They force you to slow down and reflect — even if just for a moment.

Starting last year, I embarked on a simple experiment with my friend, Seb Paquet, who’s based in Montreal and who, like me, works for himself. We decided to check in with each other once a week for an hour over Skype. We didn’t have any agenda. We would simply use that time to have an extended checkin.

We ended up doing two phases, and it served to be tremendously valuable. Not only did it deepen our relationship, it served several practical purposes. We both had an opportunity to give each other feedback on our respective projects. We supported each other when we faced challenges and cheered each other when we succeeded. The most interesting thing to happen was that, even when things got tremendously busy for both of us, we both stayed very committed to these checkins. They were not just nice-to-have; they were helping us work more effectively.

Inspired by these results, I invited my colleagues Pete Forsyth, Rebecca Petzel, Odin Zackman, and Amy Wu — who all work independently — to participate in a similar experiment earlier this year. We wanted to experiment with ways that we — as an informal network — could achieve the same (or better) benefits that people get from working in great organizations. In particular, we wanted to be more intentional about learning from each other. We called our experiment, Colearning 2.0, a play on the coworking movement.

We explored several things that we could do together, but we settled on doing weekly pair checkins, as Seb and I had done beforehand. There was some reluctance at first. One thing we all had in common was a frequent feeling of overwhelm, and taking an hour out of our week to “just talk” seemed burdensome. Not only did everyone end up finding the experience valuable, there’s a desire to continue the practice and take it to the next level.

I recently spoke with Joe Hsueh about his recent trip to Istanbul. One of the things that struck him the most about his trip was the adhan — the Islamic call to prayer. Imagine being in a bustling city of 14 million people where every day, at prescribed times, a horn sounds and the entire city goes silent. Imagine what it feels like to have that regular moment of collective, silent reflection.

I believe that checkins could be a powerful keystone habit that helps us slow down overall, which ultimately helps us collaborate more effectively. This is a hypothesis I continue to explore.

What helps you to slow down? What’s been the impact of doing so?

The Key to Effective Learning? Soap Bubbles!

Part three of a three-part essay on facilitating group learning. See parts one, “Getting real about experiments and learning,” and two, “Documenting is not learning.”

Two months ago, I blogged about my experiment with Dharmishta Rood and the Code for America Incubator, which wraps up in another few months. The goal is to help startups — in this case, a company called PostCode — develop great collaborative habits in its formative stage. The theory is that it’s more effective to build good habits from the start than it is to try to change bad habits later.

About a third of the way into our experiment, Dharmishta and I started discussing whether or not we thought it was working. We thought that it was, but we weren’t sure whether or not the folks at PostCode would agree. “We’re missing soap bubbles,” Dharmishta asserted.

“Huh?!” I responded.

Soap, Dharmishta explained, does not naturally produce bubbles. Manufacturers add chemicals to make soap lather, even though the resulting bubbles have no impact on cleanliness. In fact, some chemicals have adverse effects, such as drying out skin.

Why do they do this? Because people associate bubbles with cleanliness. If we don’t see the bubbles, we assume it’s not working, regardless of whether or not that’s the case.

When assessing progress, how you feel matters.

If you’re learning, and you feel like you’re learning, then all is good.
If you’re learning, but you’re not feeling like you’re learning, then you have a problem. You may stop doing things that are working, because you don’t realize that they’re actually effective. This is where adding soap bubbles can be useful.
If you’re not learning, but you feel like you’re learning, then you also have a problem. There is where adding soap bubbles can be harmful.

Given this, how do you design soap bubbles into your learning processes that help, rather than harm you?

It starts by being real about what you can and can’t measure, and it quickly shifts to remembering what you’re trying to achieve in the first place. There is always an intention underlying our instinct to measure, but sometimes, the complexity around measurement causes us to forget that intention.

I’m generally trying to help groups learn how to collaborate more effectively. I want feedback mechanisms that reinforce good habits and discourage bad ones. In PostCode’s case, Dharmishta and I wanted to make sure that the team was communicating and making decisions effectively. We thought that we were seeing progress, but we weren’t sure that the team was seeing the same things we were. We were (and still are) worried that the team would discard the practices before they became habit.

We designed a monthly “assessment” to help PostCode track its progress, and when we started exploring ways to measure effectiveness around communication and decision-making, we decided to take a different approach. Rather than attempting to quantify those things (as we were doing with the other dimensions we were tracking), we decided to ask qualitative questions:

Please share up to three examples of great, effective communication over the past month.
Please share up to three examples where communication could have been better over the past month.

(We asked the same questions about decision-making.) These questions will not provide some “objective” measure of how well the company is communicating as a whole or whether or not it is improving over time. However, they will:

Encourage everybody to regularly pause and reflect
Surface people’s different perceptions of what is good and what is not
Identify problems that need fixing while they’re still fresh
Remind people to celebrate what’s going well

Most importantly, these questions will remind everybody that communication matters, and that they should be practicing consciously and with an eye to improve. The very act of pausing to answer these questions on a monthly basis means that they are. These questions act as soap bubbles, but they also help clean!

Measurement is important, but feedback is even moreso. Figuring out the right things to measure is hard, but creating feedback mechanisms is not. Soap bubbles may not give you objective indicators for tracking progress, but they will — at minimum — remind people of what they’re trying to accomplish and to be conscious of when it’s happening.

Photo by Umberto Salvagnin. CC BY 2.0.

Getting Real About “Experiments” and Learning

Part one of a three-part essay on facilitating group learning.

Last year, I went to Cincinnati to visit my sister and her family. My older nephew, Elliott, who was eight at the time, asked if I could help him with his science experiment. He was supposed to pick a project, develop a hypothesis, and run some experiments to prove or disprove it.

Elliott explained to me that earlier that year, he had participated in a pinewood derby and had lost. He wanted to figure out how to make a car that would go faster. I asked him, “What do you think would make the car go faster?”

He responded, “Making it heavier.” That seemed like an eminently reasonable hypothesis, especially coming from an eight year old. I helped him define the parameters of an experiment, and he constructed a car out of Legos and a ramp using a hodgepodge of race track parts to run his tests.

In theory, mass has nothing to do with the speed of the car. The only thing that matters is the acceleration of gravity, which is constant. A heavier car should go down the ramp at the same speed as a lighter one.

Except that’s not true either. Wind resistance counteracts the effects of gravity, which might make a lighter car go slower as a result. Then again, the aerodynamics of the car might have a bigger effect on decreasing wind resistance than mass. Then there’s the issue of both the friction and alignment of the wheels. And so forth.

Wading through all of these variables would require a carefully calibrated measurement system. Suffice it to say, Elliott’s system was not very precise. When he ran his experiment initially, the lighter car went faster than the heavier car. He dutifully proceeded to conclude that weight had the opposite effect on speed than he had theorized. I suggested that he try the experiment again just to be sure. This time, the cars took the same amount of time.

He was thoroughly confused. So was I, but for different reasons. How was I supposed to explain to him all the possible reasons for his results without delving into the intricacies of physics and engineering?

It was a ridiculous thing to expect an eight year-old to figure out, but it would have been fair to have asked of a high schooler. Science is clean that way. You can set up experiments and controls, you can meticulously account for variables, and you can repeat and replicate your experiments to build confidence in your results.

This is not the case with people.

It has become en vogue in the business world to frame knowledge work around experiments and learning. This is the essence of the Lean Startup idea, but it’s not limited to lean. I’ve been as guilty of this as anyone, and I’ve been doing it for a long time now.

But what exactly does it mean to frame people-work this way? Unlike science, you do not have laboratory conditions where you can set up replicable experiments with controls. Sure, you can come up with hypotheses, but your conditions are constantly changing, and there’s usually no way to set up a control in real-life.

How can you fairly draw any conclusions from your results? What are you even measuring? The realm of trying to assess “impact” or “effectiveness” or (to get very meta about it) “learning” tends to devolve into a magical kingdom of hand-waving.

The reality is that experimentation without some level of discipline and intentionality is just throwing spaghetti against the wall. The worse reality is that — even with all the discipline in the world — you may not be able to draw any reasonable, useful conclusions from your experiments. If your ultimate goal is learning, you need more than discipline and intentionality. You need humility.

In The Signal and the Noise, data analysis wunderkind Nate Silver points out how bad humans tend to be at forecasting anything reasonably complex — be it political elections or the economy. There are way too many variables, and we have way too many cognitive biases. However, we are remarkably good at predicting certain things — the weather, for example. Why can we predict the weather with a high degree of certainty but not things like the economy?

There are lots of reasons, but one of the simplest is accountability. Simply put, meteorologists are held accountable for their predictions, economists are not. Meteorologists are incentivized to improve their forecasts, whereas economists generally are not.

What does this mean for groups that are working on anything complex and are trying to learn?

First, be intentional, but hold it lightly. Know what it is you’re trying to learn or understand, and be open to something else happening entirely. Measure something. Be thoughtful about what you measure and why.

Second, be accountable. Track your learning progress. Review and build on previous results. Be transparent about how you’re doing. Don’t use “experiments” as a proxy for doing whatever you want regardless of outcome.

Third, be humble. Despite your best efforts, you may not be able to conclude anything from your experiments. Or, you might draw “convincing” conclusions you might validate again and again, only to discover that you are totally, entirely wrong.