The Infoq Podcast

Rossen Stoyanchev talks to Wesley Reisz about blocking and non-blocking architectures, upcoming changes in Spring including Spring WebFlux, the reactive web stack in Spring framework 5, due this summer. He also discusses the differences between rxJava and Reactor. Why listen to this podcast: - Spring Framework 5 is due to be released June 25 2017 - Spring Web Flux provides a web programming model designed for asynchronous APIs - Back-pressure is important in a server environment; less so within a UI environment - It’s possible to use a Spring Web Flux client within a Spring MVC applciation - Managing sets of thread pools is more complicated than having a scalable asynchronous system More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2pPgq0G You can subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitte

Why listen to this podcast: - Using a compiler to catch errors at compile time instead of at runtime means much easier refactoring of code. - Incrementally replacing small parts of an existing JavaScript application with Elm is a safer strategy than trying to write an entirely new application in Elm - Elm packages are semantically versioned and gated by the publishing process, so minor versions cannot remove functionality without bumping the major version. - The UI in an Elm application results in messages that transform the immutable state of the application; this allows a debugger to view the state transitions and the messages that triggered them, including record and replay of those messages. - Elm has been benchmarked as being faster than Angular and React whilst being smaller code, which is attributed to the immutable state and pure functional elements. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2qmS2CT You can also subscribe to the InfoQ newsletter to receive weekly u

Jean Barmash is Director of Engineering at Compass, Founder & Co-Organizer, NYC CTO School Meetup. Live in New York City. He has over 15 years of experience in software industry, and has been part of 4 startups over the last seven years, 3 as CTO / VPE and one of which he co-founded. Prior to his entrepreneurial adventures, Jean held a variety of progressively senior roles in development, integration consulting, training, and team leadership. He worked for such companies as Trilogy, Symantec, Infusion and Alfresco, consulting to Fortune 100 companies like Ford, Toyota, Microsoft, Adobe, IHG, Citi, BofA, NBC, and Booz Allen Hamilton. Jean will speak at QCon New York 2017: http://bit.ly/2nN7KKo Why listen to this podcast: - The Compass backend is mostly written in Java and Python, with Go increasingly a first class language. The main reason for Go being added was developer productivity. - The app is based on a Microservices architecture with around 40-50 services in total. - Binary RPC, originally Thrift

Eric Horesnyi, CEO @streamdata.io, talks to Charles Humble about how hedge funds are applying deep learning as an alternative to the raw speed favoured by HFT to try and curve the market. Why listen to this podcast: - Streamdata.io was originally built for banks and brokers, but more recently hedge funds have begun using the service. - Whilst Hedge Funds like Renaissance Technologies have been using mathematical approaches for some time deep learning is now being applied to markets. Common techniques such as gradient descent and back propagation apply equally well to market analysis. - The data sources used are very broad. As well as market data the network might be using, sentiment analysis from social networks, social trading data, as well as more unusual data such as retail data, and IoT sensors from farms and factories. - By way of contrast High Frequency Trading focusses on latency. From an infrastructure stand-point you can play with propagation time, Serilization (the thickness of the pipe), an

Greg Murphy is the COO of Gamesparks, a cloud-based platform providing and a rich mobile back-end service for game developers to engage with their users. Greg takes us inside Gamesparks discussing the architecture, machine learning and what it’s like to launch in the China market. Why listen to this podcast: Gamesparks Engagement Engine Tuning the Gaming Experience The Architecture SDK’s and Real-time Data Transfers Server-side Scripts Managing Noisy neighbours and Security The Developer Experience Deploying Across Three Cloud Providers Machine Learning The China Market Notes and links can be found on: http://bit.ly/2neCjEV You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Wesley Reisz talks to Slava Oks, who has worked at Microsoft for over 20 years on flagship products, including SQL Server. He also led the kernel team who worked on the Midori operating system. More recently, he has worked on bringing SQL Server to Linux. Why listen to this podcast: - Microsoft SQL Server runs on Linux through a containerised approach called Drawbridge - Drawbridge implements a Linux loader and a minimal set of ABI calls to allow an in-process NT user mode kernel to run - SQL Server runs on top of a SQL platform layer (called SQL OS) that could be ported to run on Drawbridge - SQL Server had supportability commands added to allow the state of the system to be measured with SQL calls - A number of efficiency gains were applied to both the Drawbridge components and the SQL Server code to bring performance to within 20% of the equivalent process running on Windows Notes and links can be found on: http://bit.ly/2kVfatX Drawbridge ABIs Security LibOS Linux SQL OS Supportability Some assem

Jonas Bonér, CTO of LightBend and creator Akka, discusses using Akka when developing distributed systems. He talks about the Actor Model, and how every Microservice needs to be viewed as a system to be successful. Why listen to this podcast: - Akka is JVM-based framework design for developing distributed systems leveraging the Actor Model - an approach for writing concurrent systems that treat actors as universal primitives and the most successful model with abstraction has been streaming - Circuit breakers in Akka are a backup and retry policy; they protect you by capturing failure data and allow you to roll back - Every Microservice needs to be viewed as a system, it needs to have multiple parts that run on different machines in order to function and be fully resilient - thus is a Microsystem - Two different trends have emerged when it comes to hardware and environments: one is the trend toward Multi-core, the is a movement toward virtualized environments and the cloud - Saga pattern of managing lo

Peter Bourgon discusses his work at Weaveworks, discovering and imlemeting CRDTs for time-stamped events at Soundcloud, Microservices in Go with Go Kit and the state of package management in Go. Why listen to this podcast: - We’ve hit the limits of Moore’s law so when we want to scale we have to think about how we do communication across unreliable links between unreliable machines. - In an AP algorithm like Gossip you still make forward progress in case of a failure. In Paxos you stop and return failures. - CRDTs give us a rigours set of rules to accommodate failures for maps, sets etc. in communication that result in am eventually consistent system. - Go is optimised to readers/maintainers vs. making the programmers’ life easier. Go is closer to C than Java in that it allows you to layout memory very precisely, allowing you to, for example, optimise cache lines in your CPU. - Bourgon started a project called Go Kit, which is designed for building microservices in Go. It takes inspiration from Tiwtte

In this week’s podcast Wes Reisz talks to Neha Batra, a software engineer at Pivotal Labs. Neha spoke about pair programming in her recent QCon San Francisco 2016 presentation, and has taken time to discuss techniques to get started with the practice as well as tips for implementing it on your team. Neha also touches on vulnerability based trust and how it can help effectively build a trusting team environment. Why listen to this podcast: - If you successfully start with pair programming, other tenants of XP are pulled along with you - Ways to get creative with remote pairing to make it work - The daily retro - Overcoming hesitance with managers when trying to implement pair programming full time - Vulnerability based trust building Notes and links can be found on: http://bit.ly/2i2a0sJ How has Pair Programming Evolved Over the Years? 6m:17s - A lot of the fundamentals are the same, but with XP we take it to the extreme to be able to do it eight hours a day. 6m:24s - To pair for eight hours a day w

In this week’s podcast, Robert Blumen talks to Oliver Gould at QCon San Francsico 2016. Oliver is the CTO of Buoyant where he leads open source development efforts. Prior to Buoyant he was a Staff Infrastructure Engineer at Twitter where he was technical lead on Observability, Traffic, Configuration and Co-ordination teams. Why listen to this podcast: - Stratification allows applications to own their logic while libraries take care of the different mechanisms, such as service discovery and load balancing - Cascading failures can’t be tested or protected against, so having a fast time to recovery is important - Having developers own their services with on-call mechanisms improves the reliability of the service; it’s best to optimise automatic restarts so problems can be addressed during normal working hours - Post mortem analysis of failures are important to improve run books or checklists and to share learning between teams - Incremental roll out of features with feature flags or weighted routing provi

In this week’s podcast, Thomas Betts talks with Chris Richardson, a developer, architect, Java Champion and author of POJOs in Action. Before his workshop on Microservices w/ Spring Boot and Docker at QCon San Francisco 2016, Richardson took time to discuss his ideas on how to use DDD and CQRS concepts as a guide for implementing a robust microservices architecture. Why listen to this podcast: - "Microservice architecture" is a better term than "microservices". The latter suggests that a single microservice is somehow interesting - The concepts discussed in Domain-Driven Design are an excellent guide for how to implement a microservices architecture - Bounded Contexts correspond well to individual microservices - Event sourcing and CQRS provide patterns for how to implement loosely coupled services - When converting a monolith to microservices, avoid a big bang rewrite, in favor of an iterative approach Notes and links can be found on: http://bit.ly/2hZ8TM1 11m:51s - Microservices must be loosely c

In this week’s podcast, QCon chair Wesley Reisz talks to Keith Adams, chief architect at Slack. Prior he was an engineer at Facebook where he worked on the search type live backend, and is well-known for the HipHop VM [hhvm.com]. Adams presented How Slack Works at QCon SanFrancisco 2016. Why listen to this podcast: - Group messaging succeeds when it feels like a place for members to gather, rather than just a tool - Having opt-in group membership scales better than having to define a group on the fly, like a mailing list instead of individually adding people to a mail - Choosing availability over consistency is sometimes the right choice for particular use cases - Consistency can be recovered after the fact with custom conflict resolution tools - Latency is important and can be solved by having proxies or edge applications closer to the user Notes and links can be found on: http://bit.ly/keith-adams 3m:30s Voice and video interactions are impacted by latency; the same is true of messaging clients 4m:

In this week’s podcast, Thomas Betts talks with Haley Tucker, a Senior Software Engineer on the Playback Features team at Netflix. While at QCon San Francisco 2016, Tucker told some production war stories about trying to deliver content to 65 million members. Why listen to this podcast: - Distributed systems fail regularly, often due to unexpected reasons - Data canaries can identify invalid metadata before it can enter and corrupt the production environment - ChAP, the Chaos Automation Platform, can test failure conditions alongside the success conditions - Fallbacks are an important component of system stability, but the fallbacks must be fast and light to not cause secondary failures - Distributed systems are fundamentally social systems, and require a blameless culture to be successful Notes and links can be found on: http://bit.ly/2hqzQ6K 2m:05s - The Video Metadata Service aggregates several sources into a consistent API consumed by other Netflix services. 2m:43s - Several checks and validatio

In this week's podcast, QCon chair Wesley Reisz talks to Kolton Andrus. Andrus is the founder of Gremlin Inc. He was a Chaos Engineer at Netflix, focused on the resilience of the Edge services. He designed and built FIT: Netflix’s failure injection service. Prior, he improved the performance and reliability of the Amazon Retail website. Why listen to this podcast: - Gremlin, Kolton Andrus' new start-up, is focused on providing failure testing as a service. Version 1, currently in closed beta, is focused on infrastructure failures. - Lineage-driven Fault Injection (LDFI) allowed Netflix to dramatically reduce the number of tests they needed to run in order to explore a problem space. - You generally want to run failure tests in production, but you can't start there. Start in developemnt and build up. - Having failure testing at an application level, as Netflix does, so you can have request level fault injection for a specific user or a specific device. - Being able to trace infrastructure with something li

As InfoQ previously reported in March 2016, Dropbox announced that they had migrated away from Amazon Web Services (AWS). In this week's podcast Robert Bluman talks to Preslav Le. Preslav has been a software engineer at Dropbox for the past three years, contributing to various aspects of Dropbox’s infrastructure including traffic, performance and storage. He was part of the core oncall and storage oncall rotations, dealing with high emergency real world issues, from bad code pushes to complete datacenter outages. Why listen to this podcast: - Dropbox migrated away from Amazon S3 to their own data centres to allow them to optimise for their specific use case. - They are experimenting with Shingled Magnetic Recording (SMR) drives for primary storage to increase storage density. All writes go to an SSD cache and then get pushed asynchronously to the SMR disk. - Their average block size is 1.6MB with a maximum block size of 4MB. Knowing this allows the team to tune their storage system. - Three languages a

In this week's podcast QCon chair Wesley Reisz talks to Randy Shoup. Shoup is the vice president of engineering at Stitch Fix. Prior to Stitch Fix, he worked for Google as the director of engineering and cloud computing, CTO and co-founder of Shopilly, and chief engineer at Ebay. Why listen to this podcast: - Stitch Fix's business is a combination of art and science. Humans are much better with the machines, and the machines are much better with the humans. - Stitch Fix has 60 engineers, with 80 data scientists and algorithm developers. This ratio of data science to engineering is unique. - With Ruby-on-Rails on top of Postgres, the company maintains about 30 different applications on the same stack. - The practice of Test Driven Development makes Continuous Delivery work, and the practice of having the same people build the code as those who operate the code makes both of these things much more powerful. - Microservices gives feature velocity, the ability for individual teams to move quickly and indep

In this week's podcast, QCon chair Wesley Reisz talks to Tal Weiss, CEO of OverOps, recently re-branded from Takipi. The conversation covers how the OverOps product works, explores the difference between instrumentation and observability, discusses bytecode manipulation approaches and common errors in Java based applications. A keen blogger, Weiss has been designing scalable, real-time Java and C++ applications for the past 15 years. He was co-founder and CEO at VisualTao which was acquired by Autodesk in 2009, and also worked as a software architect at IAI Space Industries focusing on distributed, real-time satellite tracking and control systems. Why listen to this podcast: - OverOps uses a mixture of machine code instrumentation and static code analysis at deployment time to build up an index of the code - Observability is how you architect your code to be able to capture information from its outputs. Instrumentation is where you come in from the outside and use bytecode or machine code manipulation te

In this week's podcast InfoQ’s editor-in-chief Charles Humble talks to Data Scientist Cathy O’Neil. O'Neil is the author of the blog mathbabe.org. She was the former Director of the Lede Program in Data Practices at Columbia University Graduate School of Journalism, Tow Center and was employed as Data Science Consultant at Johnson Research Labs. O'Neil earned a mathematics Ph.D. from Harvard University. Topics discussed include her book “Weapons of Math Destruction,” predictive policing models, the teacher value added model, approaches to auditing algorithms and whether government regulation of the field is needed. Why listen to this podcast: - There is a class of pernicious big data algorithms that are increasingly controlling society but are not open to scrutiny. - Flawed data can result in an algorithm that is, for instance, racist and sexist. For example, the data used in predictive policing models is racist. But people tend to be overly trusting of algorithms because they are mathematical. - Data s

In this week's podcast QCon chair Wesley Reisz talks to Machine learning research scientist John Langford. Topics include his Machine Learning system Vowpal Wabbit, designed to be very efficient and incorporating some of the latest algorithms in the space. Vowpal Wabbit is used for news personalisation on MSN. They also discuss how to get started in the field and it’s shift from academic research to industry use. Why listen to this podcast: - Vowpal Wabbit is a ML system that attempts to incorporate some of the latest machine learning algorithms. - How to learn ML: take a class or two, get accustomed with learning theory and practice. - ML has moved from the research field into the industry, 4 out of 9 ICML tutorials coming from the industry. - It’s hard to predict when you have enough data. - AlphaGo is a milestone in artificial intelligence. It uses reinforcement learning, deep learning and existing moves played by Go masters. - Deep Learning is currently a disruptive technology in areas such a v

In this week's podcast, professor Barry Burd talks to Shuman Ghosemajumder. Ghosemajumder is VP of product management at Shape Security and former click fraud czar for Google. Ghosemajumder is also the co-author of the book CGI Programming Unleashed, and was a keynote speaker at QCon New York 2016 presenting Security War Stories. Why listen to this podcast: With more of our lives conducted online through technology and information retrieval systems, the use of advanced technology gives criminals the opportunity to be able to do things that they weren't able to do. - Cyber-criminals come from all over the world and every socioeconomic background, so long as there's some level of access to computers and technology. - You see organised cyber-crime focusing on large companies because of the fact that they get a much greater sense of efficiency for their attacks. - Cyber-criminals are getting creative, and coming up with ways to interact with websites we haven't thought of before. - You can have very large