Webinar - Delivering Enhanced Message Processing at Scale

下载 0

ccone

发布于

1755

人观看

#信息技术

Managing 3.8 million e-prescriptions daily for more than 1 million healthcare professionals is no small feat. And, with rapid growth in the number of digital transactions and expansion of its network, Surescripts needed to replace its legacy relational database system to address a new set of data management challenges while meeting their customers’ demanding SLAs. Join us for this on-demand webinar to hear from Keith Willard, Chief Architect at Surescripts, to learn how and why Surescripts leverages DataStax Enterprise to deliver enhanced message processing at scale.

展开查看详情

1 .Delivering Enhanced Message Processing at Scale with DataStax Enterprise Keith Willard Bill Liu Matt Griffin Chief Software Architect Enterprise Architect Lead Database Engineer Surescripts Surescripts Surescripts Jie Wu Director, Product Marketing DataStax

2 . From validation to momentum 400+ $190M 500+ Employees Funding Customers Founded in April 2010 Santa Clara • San Francisco • Austin • London • Paris • Berlin • Tokyo • Sydney (Series E – Sept. 2014) 30% + 2016 World’s Best Ranked #1 in multiple operational 100 Cloud Companies database categories © 2017 DataStax, All Rights Reserved. Company Confidential

6 . Powering cloud applications • Designed to handle any failure, ALWAYS-ON no matter how catastrophic. EFFORTLESS • Take advantage of every opportunity. SCALE • Focus on what matters most to you. • Built into your application to create INSTANT INSIGHT actionable, modern experiences. 6 © DataStax, All Rights Reserved.

7 .DataStax Enterprise subscriptions DSE Standard DSE Max Data Platform For teams that need a Certified Apache Cassandra™ ✓ ✓ solution easy to build and Advanced Security ✓ ✓ effortless to scale Automatic Management Services ✓ ✓ DSE STANDARD Advanced Replication ✓ ✓ Tiered Storage ✓ ✓ Multi-Instance ✓ ✓ In-Memory ✓ Search ✓ For teams looking for an integrated data management Analytics ✓ platform with advanced Graph Option features Management and Development Solutions DSE MAX DataStax OpsCenter ✓ ✓ DataStax Studio ✓ ✓

8 . DataStax Managed Cloud (DMC) A Fully Managed, Secure Architecture • DataStax Enterprise on AWS with Managed Provisioning and Scaling by DataStax • 24x7x365 Coverage, Lights-Out Management • System Configuration and Tuning to Meet Customer Specific Requirements • Architecture Advisory Services, Guidance and Best Practices 8

9 . Why DMC Matters to our customers • Focus on business innovation • Offload operations to DataStax • Reduce risk with a secure, white glove service by experts • Grow easily as your business expands • Accelerate time-to-market • A single data layer for on-premises and public cloud • No re-write of applications against every cloud provider’s proprietary data services • Create APIs and micro-services to a consistent data layer • Choice and flexibility for hybrid architecture needs • Extend to fit your hybrid cloud architecture • Span or migrate across infrastructure models with 100% compatibility • Retain data autonomy without cloud vendor lock-in 9

10 .DataStax Enterprise in the Surescripts Context Keith Willard, Bill Liu, Matt Griffin August 2017 CopyrightCopyright © 2016 by© Surescripts, LLC. All rights 2016 by Surescripts, reserved. LLC. All rights reserved.

11 .Surescripts network Surescripts is responsible for the largest number of clinical oriented electronic transactions in the country today • The electronic prescription that flows from your doctor’s office computer to your local or mail order pharmacy PBM Claim Pharmacy Elig, Med Hx, SCRIPT Surescripts SCRIPT Surescript Core EHR Network Copyright © 2016 by Surescripts, LLC. All rights reserved.

13 .A core load of our processing pipeline  We are message centric • Each operation is encapsulated by the receipt or transmission of a message from or to a network participant  Large volume of messages • Average 50-60 million per week. ~1/2 between 10am and 2pm.  Median size message ~4k • But ranges in size upwards to 20 MB.  Write challenge of scale to write  Read problem • Frequently need to refer to a related past message to properly process and route incoming message. 13 Copyright © 2016 by Surescripts, LLC. All rights reserved.

14 .Constant pressure to improve SLA  External failures affecting the “fabric” of our data centers  Failures related to data center synchronization processes  Failures related to software mutation • Infrastructure software upgrades • Our own application software upgrades 14 Copyright © 2016 by Surescripts, LLC. All rights reserved.

15 .Architecture transitions  Past and near present • mixed Java and .NET transaction application tier heavily virtualized • Persistent tier an MQ/Oracle relational OLTP environment • Analytical/Reporting tier Oracle Exadata data warehouse • High reliability relies heavily on a dual physical data center model (Mpls and VA)  Future (cluster centric) • Similar mixed java and .NET transactions Docker or equivalent • Persistent Tier Kafka/DSE/some relational at the OLTP tier • Analytical Reporting Hadoop/Exadata • Higher reliability and scalability multiple “logical data centers” with two physically distinct Hadoop clusters. 15 Copyright © 2016 by Surescripts, LLC. All rights reserved.

16 .Logical data center  Restructure persistent tier around two themes • Horizontally, economically scalable persistence Application Pools streaming • Eventually consistent, locally clusterable WAN extended queryable database Microservice Pools  Transition from Oracle and Oracle RAC to alternative Non-Audit Relational • Horizontally scalable with a practical Data synchronization/replication infrastructure DSE Kafka Copyright © 2016 by Surescripts, LLC. All rights reserved.

17 .End game Application Application Application Application Pools Pools Pools Pools Microservice Microservice Microservice Microservice Pools Pools Pools Pools Non-Audit Non-Audit Non-Audit Non-Audit Relational Relational Relational Relational Data Data Data Data DSE DSE DSE DSE Kafka Kafka Kafka Kafka Kafka Replicator OR Hbase/Kudu Replication ZFS Replication ZFS ZFS Always 17 Copyright © 2016 by Surescripts, LLC. All rights reserved.

19 .Overall operational experience  In a word - outstanding  Read & write performance has equaled or bettered our relational OLTP tier • Now manageably scalable  One rolling upgrade since going live in November without loss of uptime  DataStax Support was very responsive • When we were affected by a base bug (CASSANDRA-12247) that caused intermittent repair failures and write timeouts  Improved efficiency • The involved DBE reports less on-going maintenance activity rather than more for equivalent Oracle infrastructure. 19 Copyright © 2016 by Surescripts, LLC. All rights reserved.

20 .Cluster performance summary (last 7 days) Write throughput, 720 writes/sec ~9.5 million/day cluster peak peak Read throughput, 305 reads/sec ~4.5 million/day Cluster peak peak Write Latency 99% latency 65 ms/write 50% latency 15 ms/write Read Latency 99% latency 79 ms/read 50% latency 38 ms/write Exceptions none Copyright © 2016 by Surescripts, LLC. All rights reserved.

24 .Use case criteria guiding choice of DSE  Stereotyped (ie. development time) access patterns  Size thoughtfully  Replication requirements  Scale capabilities  Reliability via cluster architecture • We will require a very long baseline for full answer. 24 Copyright © 2016 by Surescripts, LLC. All rights reserved.

25 .Things we think we did right, and wrong  Things we did right • Kept to the well understood wheelhouse of DSE - primary key access pattern • Paid attention to the recommendation to stay under 50% disk capacity when determining sizing • Blue/green deployment option has given us precise understanding of behavior of new versus prior infrastructures.  Things we could do better • Have not yet broadly familiarized our db staff with DSE infrastructure ▪ So far only two individuals. • We have partition keys that include identifiers generated by our trading partners. These should be unique according to business rules, but we do not enforce uniqueness, so we are at risk of wide partitions.  Things we don’t yet know • Major version/ major OS upgrade events haven’t occurred yet. 25 Copyright © 2016 by Surescripts, LLC. All rights reserved.

26 . Next steps Learn more about DataStax Enterprise: Learn more about DataStax Managed Cloud (DMC):  eBook • The Magic Number 5 - Why Cloud Applications Require Always-  Dedicated website on Data Platform • Why Your RDBMS Fails at Scale?  eBook • Time to Innovate – Introducing DataStax Managed  Analyst report Cloud • Gartner 2016 Critical Capabilities for Operational Database Management Systems  DMC datasheet  Webinar recording  Webinar recording • DataStax Enterprise Managed Cloud: Focus on • Introducing DataStax Enterprise 5.1 Innovation, not Administration  Trainings • https://academy.datastax.com/courses  Download DSE and other toolkits 26

4点赞

1收藏

0下载