Product Development in the Risk Space

For marketplaces, earning the trust of buyers and sellers requires providing consistently secure and reliable platform experiences. However this critical component of a marketplace’s business model happens to also be particularly complex. Product Development within the Risk Space is highly dynamic in nature and in its operational demands. When attackers pivot, organizations must be agile enough to react. When new vulnerabilities develop, organizations must investigate and research. And of course, when events occur, organizations must mitigate and remediate.

On June 15th at 7:35pm IST, Etsy Director of Product Management, Yi Liu and Director of Engineering, Anthony Mazzarella will take to the Dublin Tech Summit Horizon Stage for a conversation about Marketplace Risk. Join them to learn more about Etsy’s tailored approach to providing a secure and reliable marketplace experience to its 5.5M sellers and nearly 89.1M buyers. 

The evolution of Etsy’s global Risk engineering program has been influenced by learnings from DevOps, SRE (site reliability engineering), and other modern engineering concepts. Anthony and Yi will walk through a few of these unique approaches, speaking to each topic from both Engineering and Product perspectives.

One key area of learning – and one that’s probably familiar to many – is the shift from operational reactivity to proactive product development. In order to effectively move away from tactical in-the-moment responses as the default, Etsy teams have been identifying specific priorities defined by problems, outcomes, and opportunity impact. Anthony and Yi will discuss evaluating trade-offs and how they measure for qualitative and quantitative success.

 

Etsy places enormous emphasis on the empowerment of its employees. By rethinking how a business defines engineering responsibility towards delivering on risk objectives, Etsy has established a development approach through which features are designed and informed by – rather than gated by – Risk advocacy and education. Etsy’s leaders will discuss how they established this enhanced approach and the tooling that supports it today. 

 

Finally, to illustrate how these learnings are put into practice, Yi and Anthony will share a case study from their teams. They’ll walk attendees through the design and the eventual implementation of a specific product; how their teams iterated, experimented, trained and shifted ownership; on to post-launch results and potential future platform enhancements.

Interested to learn more? Join Yi and Anthony at the Horizon Stage, June 15th at 7:35pm IST, and feel free to visit Etsy’s Engineering Blog and Careers Page in the meantime. 

Used to develop the blog post

Summary

Risk is a critical area for Etsy. The experience of our buyers and sellers is dependent on the safety and reliability of the marketplace. This also influences their confidence in Etsy as a brand.

What makes Risk complex is the highly-dynamic nature of its problem area and its operational demands. As attackers pivot, the organization must react. As new vulnerabilities develop, the organization must research and understand. As events occur, the organization must mitigate and remediate.

To find success and sustainability, Etsy undertook an evolution of its Trust & Safety programs. First, by applying a product mindset. Then, by applying learnings from DevOps, SRE (Site Reliability Engineering), and other modern engineering concepts to empower more people to support and deliver Etsy’s responsibility for a safe and trustworthy marketplace at global scale.

Potential Focus Topics

  • Shifting operational reactivity to proactive product development
    • Shifting away from tactical in-the-moment response as default
    • Identifying priorities by focusing on problems, outcomes, and opportunity impact
    • Evaluating trade-offs between near-term relief vs. long-term sustainability
    • Measuring, qualitatively and qualitatively, to understand success
  • Empowerment of Etsy and scaling Risk
    • Center of Excellence vs. Community of Practice
    • Building tools that simplify and empower more people to support events
    • Rethinking responsibility within engineering towards delivering on risk objectives
    • New features are not gated by Risk, but designed and informed by Risk advocacy and education
    • Really rather not say the word DevSecOps, but can
  • Case study on RTT
    • Real-time Takedown is the first manifestation of all of the above
    • Inception: How did we conceive of empowering agents? How did we conceptualize the design?
    • Implementation: How did we iterate and experiment as we built it? How did we train and shift ownership of roles/responsibilities?
    • Growing: How are we thinking about further enhancing the platform (e.g. machine learning)? How do we plan to democratize complex technical solutions (e.g. machine learning) for non—technical users?

Questions

  • Are there any concerns that presenting our growth arch towards a product mindset exposes brand reputation risks? For example, talking about historical manual operational remediation for large volume events.
  • Given the sensitivity and risk of talking about specific vulnerabilities, exploits, etc., what degree of detail is the organization comfortable with?

Reviewed by Anthony Mazzarella, Yi Liu, Keyur Govande, Kirstin Beal, Leonel Rodriguez, Lily Cohen