Design Critique of NBA Top Shot Queue
What the queue is, how it works, and how to potentially improve it.
Disclaimer: Everything written in this article are based on my own assumptions and opinions. In no way are they affiliated with Top Shot or representative of all the testing, data, decisions and work needed to make Top Shot the way it is today. So in other words, this is for entertainment purposes only!
What is NBA Top Shot?
Started in October 2020, NBA Top Shot is an online community for trading virtual basketball cards. As a member, you are able to trade what are called moments, which can be ‘signature moves, jaw-dropping blocks, or unbelievable jump out-of-your-seat plays’ from top NBA players (Source: NBA TopShot). These moments are broken down into play categories, which are blocks, dunks, 3-pointers, handles, jump shots, assists and layups.
Currently, there are 2 ways to obtain moments:
- Users can buy packs, which are similar to physical sports cards packs, when they are announced on the Top Shot website.
- Users can purchase moments directly from other users based on listings in the marketplace. Currently the lowest price for a single moment is $8 (Bogdan Bogdanović Series 2 Layup) and the highest price is $250,000 (Lebron James Series 1 Legendary Dunk).
As someone new to the cryptocurrency and NFT space, I’ve been incredibly fascinated in this area and impressed at how advanced we’ve come in today’s technology. And NBA Top Shot seems to be quickly climbing up the ranks of blockchain decentralized apps. With over $230 million in transactions since it’s launch, Top Shot is hoping to get a slice of the pie among other cryptocurrencies and DAPPS(Source: MarketWatch).
What I really like about Top Shot is that Dapper Labs, the Vancouver-based blockchain company that launched Top Shot, officially partnered with the NBA to make this happen. As Caty Tedman, Head of Partnerships at Dapper labs stated:
NBA Top Shot is a collector experience … it’s really an environment for collectors to come and celebrate the sport of basketball.
Although I grew up watching hockey, I remember being obsessed about anything related to my favorite player, Matt Sundin. Magazine, ads, billboards, so I get the excitement of owning these digital assets that is backed by a sports league (NHL Top Shot, anyone?)
(If you want to learn more about NBA Top Shot, I highly recommend watching this video that explains how it works in more detail.)
Why I chose to critique the Queue
As a product designer and member of Top Shot, I can’t help but fully immersive myself into every area of the website: signing up for an account, joining the Discord, browsing the support topics and watching live pack openings. But admittedly, most of my time is checking my phone for the next pack drop.
I’ve been a user for over 2 months now, and it’s been an exciting journey in a digital experience that feels different. Out of everything I’ve done so far in TopShot, the queue and pack opening experience has felt the most unique and unknown for me: How do I open a pack? Will I still get something in the mail? Since it’s digital, can I just ‘un-open’ a pack if I don’t like it?
These two experiences has arguably been the product differentiator for me in the world of e-commerce. And as someone who started their own e-commerce brand during the pandemic, I fan boy about the impact and applications this can have on my store and beyond. To compare Top Shot to an e-commerce store would be widely inaccurate, but that’s the closest I can get for now. I know for certain this is more than an e-commerce site, it’s a community and environment unlike anything I’ve experienced before.
Target Users
Before we dive into the experience, let’s begin by talking about the target users of Top Shot. While browsing through Discord, YouTube and Clubhouse, I’ve been able to get a general sense of the types of users that have been active the community. None of these are meant to be mutually exclusive, meaning one target can blend into another target.
Basketball Fans 🏀- These are the people who watch live games, talk about their favorite players, and are active in the NBA community.
- Need: Enjoy cheering for their favorite players and teams, want to be part of the NBA community, enjoy the rush of a live NBA game.
- Value: A sense of loyalty and belonging to the NBA community.
People in the general sports community 🏅- People interested in other sports and were tangentially aware of NBA Top Shot from the sports community.
- Need: Enjoys watching sports.
- Value: A feeling of surprise and disruption from other sports.
People new to digital assets 🌐- Friends and family of a Top Shot user, generally spread through word of mouth.
- Need: Want to learn more about what they are.
- Value: A novel experience at an affordable price.
Scalpers 🤑- People in it for a quick buck.
- Need: Make profit with a quick flip, doesn’t care about the sport.
- Value: Feeling good about making quick money while being in control.
Investors 📈- People looking to acquire more appreciating assets besides stocks, real estate, and crypto.
- Need: Diversification of wealth
- Value: Not having to worry about having only one source of income.
Given that Top Shot is still new, they are most likely focusing on growth in acquisitions and retention. One interesting feature is that there is currently a withdrawal time of 4–8 weeks. While this may be based on volume and high demand, I think this is favorable for the company to retain their users in order to keep trading their cards with any available balance.
Assumption 1: While Basketball fans may be the most loyal to Top Shot, from a business standpoint they most likely aren’t hard to be acquired or retained. Even if they are frustrated with withdrawals, with enough marketing from the NBA or their favorite players, users will come back. I imagine a future where Top Shot gives away moments during their half-time breaks, or even packaging merch/food bundles with moments.
Assumption 2: I’m not sure how well Dapper Labs is dealing with scalpers, but in terms of 1) Implementing restrictions when packs are dropped 2) Moment purchase requirements for some packs and 3) Restrictions on account openings during pack drops, I would say the team is addressing it pretty well.
Assumption 3: Finally, for general fans of the sports community, I would assume there’s a pretty high drop-off rate from acquisition to retention. While they may be interested in Top Shot, they’re in it less for the experience of getting a pack but want to browse. (Similar to someone hearing about and seeing where the suite section is at a sports game without purchasing and experiencing it themselves).
Therefore, I’ll be focusing the queue experience on people new to digital assets who really want to experience what it’s like to own an NFT. These people are most likely to continue in the product funnel and are an important group for TS’s future growth. This sounds counter intuitive considering they may not be fans of basketball but I think there’s tremendous opportunity to introduce them to the sport in addition to NFT’s.
I want to emphasize again that this is all based on my assumptions, but I hope this helps you understand the mental model I’m coming from.
How the Queue Works
Currently, the only way to purchase Top Shot packs is when they are announced. Part of this excitement and feeling of exclusivity are that packs can be announced at any day, usually in the afternoons or at night (typically they give more than 8 hours notice and to also accommodate for time zones).
To quickly summarize Top Shot’s explanation, this is how the queue works:
- In the time leading up to a pack drop, users (who are signed up to the platform) are notified via email. Alongside this, Top Shot sometimes also goes live on Twitch which can serve as another notification that features guests, NBA players, and pack openings to hype up the community.
- Users visit ‘nbatopshot.com where you’ll be able to access the pack by selecting the thumbnail of the soon-to-be-dropped pack on the site’s homepage.’
- And once the drop is live, ‘collectors should refresh their screens and click the button to “Join the Drop.” There will be a buffer of time for all collectors to join the waiting room. Sometimes this buffer will last for 15 minutes before the scheduled time.
- Users be added to a waiting room with all of the other collectors hoping to get a pack.
- The system will take ~60 seconds to randomly assign each collector in the waiting room a place in the queue. In cases when we sell out, your place in the queue will largely determine whether you’ll be able to purchase a pack or not.
In the past 2 months, I’ve been able to secure 3 total packs without any issues. However, with as much as 200–300k people lining up, there are bound to be issues.
Design Principles
The queue experience has always reminded me similar to a food delivery experience, where I try not to stare at it too much. With that being said, the queue still a first time user experience. Let’s put ourselves in shoes of someone new to digital assets. They might think:
“I’m lining up among what seems like hundreds of thousands of people to pay money for a video clip of an NBA player.”
“The chances of me getting a pack are slim. However, I will stay in line because my family and friends care are passionate about this.”
“I’m not sure what to expect but this feels new to me and I look forward to what’s going to happen.”
“Is this even legitimate?”
To help guide us, I’ve developed a list of general good design principles to follow for first time user experiences:
- Make it clear what you want me to do.
- Tell me what to expect and give me feedback.
- Minimize friction and reduce effort to a minimum for me.
Confidence in Getting a Pack
So far, a few things are pretty clear:
What makes users stay in the queue
- Knowing they will get a pack.
- Hoping significant people will drop off so that they can get a pack.
What makes users leave the queue
- Knowing they won’t will get a pack
- Users who already got a pack but are curious to see what queue number they get.
For users who are in the queue and think they can get a pack, they most likely won’t drop-off. It’s a bit hard to benchmark Top Shot’s data with drop-off rates, but based on my experience joining pack drops, I would estimate 1–3% of users who are able to get a pack drop-off from the queue, mainly due to payment or internet issues. I’ve include payment issues as part of drop-off because although that may be related to the check-out flow, not being to pay will cause the queue to move forward, so I’ve included it as a factor.
This means that if a drop has 36,000 available packs, a user should stay in the queue if they get a queue number ranging from 36,000- 37080. Obviously, users won’t make such calculations, and I’ve seen folks recommend staying in the queue even if they are 5k over the pack limit. In this example that would mean staying in line if you are 41,000 or under. I would say this is a pretty slim chance of securing a pack.
Top Shot clearly does not have a problem with selling packs. In fact, their real risk might be selling so many packs that the marketplace becomes diluted, slowly diminishing the feeling of exclusivity of acquiring moments. That would only happen they drop daily packs for months, or dramatically increase serial numbers for each moment.
Queue Mental Model
The Top Shot queue isn’t a first-come first serve model until sold out like a traditional e-commerce store. That only happens after users are being assigned a ‘randomized’ number. And that’s fairly done to prevent the same people from logging in early and grabbing all the moments. For the most part, random queue makes sense given the volume of people. But this can cause uncertainty. Here are some quote from users:
Man. What is their “random” queue algorithm?? Twice now it’s put me at > 215,000!!! So frustrating. — CryptoBaller
What’s the algorithm for randomizing in queue…. its definitely off as to I have yet after 6 months get a rare pack- gionitus300
“Have they ever published how they “randomize” the queue?” — TheSmithSociety
Around pack drop times, I also went to Twitter to see what folks were talking about:
This sense of novelty and scarcity is part of the emerging market of NFT’s, and as Mark Cuban states, “Digital goods, Cuban argued, are just as valuable as tangible physical goods, and operate on the same economic principles of supply and demand.” And as a Top Shot user, this makes me even more excited about a metric of scarcity, and the experience of obtaining a moment can make me feel part of an exclusive tribe.
And somewhere within that experience lives the queue. Ah, the final boss. While I agree that the existing queue experience is most likely the best short-term solution to combat bots and ensure equity, the opportunity lies in how users feel about the random queue experience.
Users not getting a pack = Frustrated users on Discord questioning the random queue experience.
Based on data collection, our problem statement is that:
As a user, I want to make sure I an entering an un-biased queue, so that I have a fair chance at purchasing a pack.
How Might We make users more confident about random queue pack drop experience?
Like any good experiment, we must develop good hypotheses. Ultimately, anything that is tested (our independent variable) needs to move the needle on a metric (our dependent variable). It’s important to call out the ultimate goal of address our painpoint, which, like our HMW statement proposes, makes users feel more confident about the random queue experience.
Our dependent variables include:
- Increased NPS scores on the product.
- Less volume on Discord and Support Center tickets about random queue.
- Increased brand loyalty.
For our independent variables, I propose a few high-level concepts to test:
Queue Randomizer Animation
What It Is: Just like an online drawing for an Instagram giveaway or lottery, once users enter the queue they see numbers being randomized and slowly stopping, landing on their queue number.
Why this idea is effective: While a randomizer may seem like a simple idea on the back-end, users experience something else. From gathering more data on Discord and Twitter a few things stood out:
- Users believe the queue favors NBA players and celebrities.
- Users believe the queue favors those who have a balance on Dapper.
- Users believe the queue favors those who have more moments.
How might we remediate that? Research has shown that people judge a book by it’s cover, meaning that our brain uses visual cues to help interpret something. As it currently stands, the queue number shows up static. However, along with color and shape, the use of motion and depth can help us perceive and interpret exactly what we are seeing.
I hypothesize that if Top Shot uses animation before a user knows which number they get, then users will be able to more fairly judge what’s in front of them that can better contextualize any emotional reactions from their queue number, whether large or small.
Sound
What It Is: Users are able to hear non-diegetic sound while their queue number is being generated: A riff, dance music, or indication of celebration.
Why this idea is effective: Top Shot has done an awesome job at consistently incorporating music, an EDM trap-like genre that plays during pack openings. The music choice is a great combination of the excitement of the environment and demographics of users who would use Top Shot. I believe there is a big untapped opportunity with Top Shot with sound that can be used more prominently in the E2E experience that can replicate that of an in-person live game, for example.
What Is Non-Diegetic Sound?
In filmmaking, non-diegetic sound, also called commentary or non-literal sound, is any sound that does not originate from within the film’s world (Source: Masterclass). An example would be a record scratch sound added for comic relief is not heard by the characters in the film.
In a digital world, Top Shot can use a similar type of sound when the drop is live and users are in a queue. A short, non-linear sound such as a riff or build up and drop of a beat can imply a sense of movement from the system (queue page).
What happens if we don’t add sound? As stated in the Psychology of Music in Gambling Environments (Not saying Top Shot is gambling, but there are parallels with random queue), researchers speculate that the absence of music in random numbers betting can:
- Limit arousal
- Put more focus on the loss for the gambler (i.e., the lack of soothing auditory stimuli heightens the loss feeling). Music would be likely to reduce negative affect experienced by players through cognitive regret and frustration (i.e. when a user can’t get a pack).
I want to finally call out that using persistent music throughout the queue time will most likely increase drop-offs from the queue, since users are most likely multi-tasking while they wait for the drop.
I hypothesize that if Top Shot uses sound in the queue with purpose and intention then we can create familiarity which allows users to persevere and more willingly accept the random queue system.
Social Proof
What it is: Once users know their queue number, they will be able to see their place in line relative to other users.
Why this idea is effective: The Solomon Asch conformity experiment in 1951 is a classic example of how people were more likely to conform to a group’s decision. To quickly summarize it, Asch gathered male college students to participate in a line judgement task. Each group was shown images like the ones below:
They had to state which line (A,B or C) matched the target line on the left. However, only one of the people in each group was actually being tested. The other seven agreed beforehand what their answers would be, which the real participant had no idea about. During those 12 tests, 75% of the unknowing participants conformed and gave the wrong answer at least once.
When Asch asked the unknowing participants why they conformed, he found that people follow social proof for one of two reasons:
- They want to fit in with the group.
- They believe the group is better informed than they are.
The herd mentality can create the feeling of being a part of a community, which increases confidence in the queue randomizer.
I hypothesize that if Top Shot shows a user’s queue position relative to other users then they can trust the group mentality that they are “in this process together.”
Risks
If it ain’t broke, don’t fix it .Right now, the queue works. And I admit it may not be the most high impact feature to work on. From the business standpoint, revenue from purchase conversions will be top line metric, with retention of users a close second. Implementing any of these concepts could potentially create negative cognitive friction (or it could be positive). I don’t have clear data indicating that the queue doesn’t work, and that’s a good thing.
If Top Shot want to continue innovating in this space and being leader in decentralized apps and beyond, incremental changes like these can create a bigger movement that addresses the larger question of how consumers acquire digital assets online. Given that TS is new, a lot of it is psychological, which have large impacts in a user’s perception of the brand.
Next Steps
While these are not the only solutions or highest impact problems to work on, they serve conversation starters as the brand continues to iterate and learn within their beta version. And the great thing is, I believe that Top Shot has already been addressing some of these issues, whether they know it or not.
By bringing the Top Shot team to join a queue live on Twitch when a pack drops, they are addressing the confidence problem of a user potentially thinking random queue favors particular people.
Caty Tedman, head of partnerships at Dapper labs, discussed the biggest obstacle to widespread adoption of NFT’s:
Patience! It took us a long time to get right for fans and the league and build the right technology around it. I think bad execution will set us back and good execution has the power to accelerate multitudes.
The team knows that they’re doing And as I’m writing this article, Dapper Labs just acquired $305 million in new funding (Source: WSJ)! With investors from NBA legends and celebrities like Will Smith and Shawn Medes, I think Top Shot is here to stay. I’m excited to see how the tech and design will continue to evolve in delivering an even more incredible customer experience.
