Work Product Feedback
This is the type of feedback team members here regularly receive
We like to joke that the most commonly asked question at Clipboard is “Where’s the doc?” Not only do we use written artifacts to drive work forward, but we also provide frequent written feedback on those artifacts. We then use grades in our written feedback to transmit just how far above or below standard the work at hand is (in the eyes of the author’s manager).
We believe that frequent written feedback helps us both develop teammates and assess coaches. For the former and the latter, you’ll frequently hear us discuss “mileage on feedback”. If someone has strong mileage on feedback, it means they’re able to internalize feedback well and incorporate it into future work. The presence of regularly written feedback (which is a function of manager discipline) gives us something to grab a hold of when we retrospectively think about a team member’s mileage on feedback.
Here’s a sample of some real written feedback on pieces of work product from our team.
Feedback #1
I thought yours was excellent. Rough grade would be 8/10.
A couple of things come to mind w.r.t. how you could improve (mostly around how we message specific portfolio decisions):
There's an implicit prioritization of your work that outlines "within 'quality' we're going to explicitly focus on 'reliability' and worry about the other components later"
In other words, reliability is just one piece of the ideal impossible experience that we want to create for facilities -- I'd share your thinking as to why you think we should approach these component by component instead of attempting to "solve quality"
(FWIW I agree with your approach, but there's a valid argument on the other side)
Related to the above, the "new HCP" hole that Attendance Score creates is something that (in theory) basket size would correct for -- I'd dig into the data to size this problem
Our investment hypothesis has been that if we
Reduce the number of late cancels and
Make Urgent Shifts excellent
Then we can solve the most acute reliability problems through Urgent Shifts
What are we going to do if this ends up being wrong? I assume new solutions would jump the priority stack -- ideally would outline to the reader in the cut list an investment that could jump up if we end up being wrong
What are we going to do if we end up being right?
You aptly point out re: solely relying on refill metrics --> "This is missing the “anxiety” component as a shift can have many cancellations against it, which creates a poor reliability experience, even if we fill the shift"
Do we continue investing in getting the cancellation number down? What's our target number s.t. we move on to a new problem?
…
Feedback #2
I wanted to deliver feedback on your recent document:
Overall, I think the doc is at a ~4 out of 10. Areas for improvement:
Hypothesis testing / metrics. I think we can organize this more clearly + be more rigorous. Here are some guiding questions:
First, I'd articulate the benefits of the project to the business. What are your primary vs. secondary benefits?
Next, I'd articulate the costs (i.e. the expenses + time spent)
Given those costs, what benefits do you have to see to make it worth it? How measurable / attributable are these benefits (i.e. should we just take the benefits to be true on faith)?
Answering this question maps your success metrics to your rollout. Given your phased release, you can now say, "If I don't see X after the first phase, I should just kill the project."
As an aside, I think a few of the metrics you proposed are actually quite hard to attribute to a specific cause. I'd be comfortable with you saying, "I think some benefit will accrue to this metric, but we'll take that to be true on faith"
Are those benefits realistic given what we've seen from our prior tests?
Risk assessment.
You identify some risks that would reduce the benefit (e.g. low activation rate). Should we track that as a success metric?
Separately, especially since this is an operational motion, I think you should also consider execution risks (not just product / customer risks).
This is particularly true for experiments like this, where execution errors might be rare, but when they occur, they are extremely painful for individual customers
Hope this is helpful! Looking forward to reviewing a second rev.
…
Feedback #3
I realize I haven't delivered grades in a few weeks, so just wanted to fire off some quick feedback.
Doc for team member - 8/10
What was noticeable here was the difference between v1 and v2. Your v1 was a 4/10 in my mind, specifically just thinking about the need to simulate how some of that content would've been received. But you did an excellent job absorbing that feedback and turned this doc around into something that I genuinely think will excite him and get his development back on track.
Just like how we anticipate scrutiny when writing WBDs, I'd try to anticipate reactions when delivering content to directs (be thoughtful about packaging to get the outcome you want).
Portfolio - 7/10
The state of the business was excellent but we struggled with the thesis. Your v1 was probably a 4/10 (mostly a function of writing quality and clarity). I think I'd expect to see larger steps of improvement next time when working through feedback -- it seemed like we were addressing some gaps in some places but other parallels were left open. I recognize that a) the "thesis" is inherently fuzzy and b) you had a backlog of docs you were itching to get through. I know you were being hard on yourself, too.
Potentially helpful writing tips to potentially leverage the next time: i) read it aloud (makes the repetitive words very obvious), ii) either make a copy of the doc and start a scratchpad or just start from scratch.
…
Feedback #4
I think we will need another rev on this Learnings Document. I would grade this at a 3/10. There is some good stuff here; the headline is that I struggled to get clarity on a) what did we learn/see from the experiment that is most important (and can be tied back to the motivation in the brief) and b) armed with that information, what should we do.
Here’s one idea on how to more clearly lay out the document:
Analysis on the numbers (2-4 pages).
Lets make sure we go to the right level of depth in our analysis so there is power to our statements, but we keep the reader focused on the meat that answers if InstantFill deserves to exist.
^ needs to be clear in our writing; I had a hard time interpreting some of the graphs and tables (or if there is context on the experiment that I was missing that helps shape how I should read the document).
Customer Conversations (1-2 pages). this helps create a better mental model over the intermittent needs claim and behavior/workflow around posting/deleting shifts so quickly, even when they were filled so fast. I am craving some insight here.
Anecdotal examples (as many pages as you think is helpful)
This was excellent. Great job going deeper to see for yourself.
Hope we can get another review between just you and I by Wednesday? Want to make sure we are confident in the v2 document before sharing more broadly. Let me know if the timeline is too soon considering what else is on your plate.
Excited!