RabbitMQ: What’s New & Changing after 10 Years of Application Messaging?

Cross posted from The Pivotal Blog.

10 years ago, RabbitMQ was first released to open source. Since that time, RabbitMQ has grown to become the most widely deployed open source message broker.

Whether you’re familiar with RabbitMQ or just learning, you’ll want to tune into this webinar with Daniel Carwin, development manager for the RabbitMQ team. Learn about the latest capabilities of RabbitMQ, and hear the future vision of how it will evolve to meet tomorrow’s application needs.

We’ll cover the following:

• Brief history of RabbitMQ
• Core design principles and how they help today’s applications
• Common use cases and patterns
• Your questions answered Latest features of RabbitMQ:
• Messaging
• Language support
• Distributed deployment
• Enterprise & cloud support
• Management and monitoring
• Future vision and roadmap for RabbitMQ

Presenters: Daniel Carwin, RabbitMQ Development Lead, Pivotal & Greg Chase, Pivotal

Watch the webinar: https://content.pivotal.io/webinars/rabbitmq-whats-new-and-changing-after-10-years-of-application-messaging?utm_source=pivotal-brighttalk&utm_medium=webinar-link&utm_campaign=10-years-of-rabbitmq_q117

RabbitMQ: What’s New & Changing after 10 Years of Application Messaging? from VMware Tanzu

Apache Geode Graduates to Top Level Project in Apache; Up Next: Microservices

Cross posted from The Pivotal Blog.

Just eighteen months ago Pivotal granted over 1 million lines of code from the Pivotal GemFire code base to The Apache Software Foundation (ASF) to help create the Apache Geode project. We made this decision because we saw many of our enterprise customers gravitating towards open source software-based solutions as part of larger IT modernization efforts. These customers understand products based on open source projects often evolve faster than their closed-source counterparts, provide more roadmap and release transparency, and reduce the risk of vendor lock-in. We couldn’t agree more.

Apache Geode began as a podling in the Apache Incubator project. Since then, an active community of contributors has grown around the project, helping improve and add capabilities to the in-memory data grid. Given the role in-memory data grids and event-driven architectures play in building cloud-native applications, the interest in Apache Geode is no surprise.

It’s been an active and fruitful year for Apache Geode. The first Apache Geode Summit was held in March 2016 and drew close to 100 community members from enterprises including Bloomberg, Southwest Airlines, Murex and TEKsystems, among others. In October 2016, Apache Geode 1.0 was released, a major milestone for any open source project. And now, the Apache Geode community has achieved an important milestone! Pivotal is thrilled that just this week the ASF graduated Apache Geode to a Top-Level Project (TLP), “signifying that the project’s community and products have been well-governed under the ASF’s meritocratic process and principles.” We extend a hearty congratulations to the Apache Geode community, without whom TLP graduation couldn’t have been achieved. This truly was a community effort, representing the best of open source and the “Apache Way”!

Of course, Apache Geode’s graduation to TLP is just one step in a longer journey to help enterprises across industries support mission critical applications with a modern, open-source based, in-memory data grid. This includes supporting an event-driven and microservices-based approach to application architecture. Microservice are autonomous across a network boundary, and inter-microservice communication is best handled through an event-driven approach – a sweet spot for Geode.

Geode and Microservices

Over the last few years, a microservices-based approach has emerged as the ideal way to build web-scale, applications, and more and more enterprises are looking to adopt this pattern for their custom software. The benefits of this approach are compelling, given the availability, scale, and speed that companies like Netflix have achieved.

Here at Pivotal, we’ve been at the forefront of the rise of microservices, with Spring Boot as the de facto Java framework, Spring Cloud Services based on Netflix OSS, and Pivotal Cloud Foundry for continuous integration and delivery. So, how does Geode fit in with this movement?

It turns out Geode is a perfect complement to a microservices architecture thanks to joint development work from the Spring and GemFire teams, the results of which have been extended into Spring Data Geode. In terms of microservices, there are implications for how data is managed when breaking applications down into specific, autonomous services that can be developed and scaled independent of one another, including how data is communicated and shared between different microservices.

From an architectural perspective, an event-based architecture is ideally suited for interactions between these loosely coupled microservices. Fortunately, Geode was designed for event-based architectures and naturally supports a microservices application development approach. The Geode Continuous Query feature, for example, can be used within each microservice for specifying state changes the microservices wants to subscribe to. The events that are of interest to a microservice can be easily specified using OQL. This convergence of features in Geode that fulfill the requirements of microservices is no accident – it is part of Geode’s design. The blazing fast performance of in-memory computing, coupled with an event-based architecture, make Geode an attractive choice for supporting microservices-based applications.

Geode also benefits from over ten years of development (as GemFire), maturing within numerous enterprises, and now advancing through community-based, meritocratic development. The seemingly daunting data challenges of microservices can be met without compromising the benefits of microservices.

Again, a big congratulations to the Apache Geode community for achieving this historic milestone! We at Pivotal look forward to continuing to work closely with the community and Pivotal customers to move the project forward. The sky’s the limit for Apache Geode! Interested in learning more about Apache Geode? Check out this five minute Geode tutorial, ask questions on the Geode mailing lists or on StackOverflow, or download Geode here.

Spring One Platform: Culture vs. Code or Culture as Code

Cross posted from my LinkedIn article.

It’s really telling when the first hour of one of the geekiest developer conferences in the industry, Spring One Platform, doesn’t show a single line of code. Instead much of the discussion was about culture and dysfunctions in culture.

Keynoters we heard from in the first hour of the first day include:

Onsi Fakhouri who discussed about the divides between developers and operators.

Cornelia Davis who discussed the gender gap that exists in the software industry these days, and how we can fix this. View the trailer for Debugging the Gender Gap for a great documentary about this topic. (Note: Cornelia actually did present the first lines of code at the conference – 4 lines of her first BASIC program as a middle school student).

Bridget Kromhout who made the point that artificial technical boundaries such as containers won’t fix a broken culture.

Phil Webb who mentioned of the divisions between DBAs and operators and developers in an IT department.

Pointing out cultural dysfunction in IT is nothing new in the industry.   Salesforce made a lot of hay in its earlier days by promising “no software” – just bypass the IT department altogether. That was the beginning of chapter 2 of the “shadow IT” narrative where developers and business users procure their own SaaS and cloud services without asking permission from a centralized corporate IT function.

A funny thing happened in the industry, however. Custom software didn’t go away. It’s now much more pervasive and important to companies. It’s also become a source of differentiation. Bypassing IT didn’t make software development better.  Now, companies are looking to the corporate IT function to better enable their developers to deliver unique business value.

The various technologies and practices discussed at Spring One Platform have a lot to do with mitigating and bridging these company cultural gaps. Onsi’s talk, which you can view an earlier rendition on video, does a nice job of explaining how we might all get along in one happy software collaborative lifecycle.

Now culture can be inscribed into the code that automates the innovation lifecycle.

Geode Summit Takeaways: User Stories, New Architecture, and Futures

Cross posted from The Pivotal Blog.

If you missed the amazing, first Apache Geode Summit held this past March, never fear, all of the slides and video replays of the sessions have been posted for your viewing pleasure.

We can describe this content in just a few words—amazing talks, excellent speakers, deep experience.

And, here are three big things you can learn by checking out the replays:

• #1: Apache Geode Scales Mission Critical Financial Services & Travel Apps
• #2: Geode Powers Modern, Data-Driven Architectures
• #3: Geode Has a Promising Future Ahead

Each of these are 30 minute sessions. So, it is easy to make some time and check them out. Microwaving a bowl of popcorn before viewing is recommended to maximize your enjoyment.

Takeaway #1: Geode Scales Mission-Critical Financial Services & Travel Apps

We heard from a number of production users about their high scale, high performance applications which take advantage of Geode’s distributed in-memory processing.

Combining Stream Processing and In-Memory Data Grids for Near-Real Time Aggregation and Notifications

First, Olivier Mallassi presented how Murex uses Geode in their risk management solution for financial services.  In his case, Apache Storm and Geode are integrated, even collocated, and applied to the fast processing side of a lambda architecture. While Geode is used for immutable events and event logging, one of the most powerful features he covers is continuous queries. Developers can think of these as a distributed, scalable set of in-memory listeners which use data to identify and push event notifications to clients.

Slides can be found here.

Wall St. Derivative Risk Solutions Using Geode

Then we heard from Andre Langevin, currently with Canadian Imperial Bank of Commerce, who has built several such systems using the technology in Apache Geode.  In his talk, he shares a series of insightful architecture whiteboards, explaining how Geode is used as a data  AND compute grid, rolling up financial risk across multiple trading systems.

Slides can be found here.

Design Tradeoffs in Distributed Systems—How Southwest Airlines Uses Geode

Finally, we heard from Brian Dunlap of Southwest Airlines. He explained just how Geode provides real-time intelligence, integrating data from reservation feeds, checkins, kiosks, boarding, gates, flights—basically from everything that moves. After explaining how his system supports critical operations and decisions, regularly crunching out optimizations for over 1,000,000 schedules, he shares many of the lessons learned

Slides can be found here.

Takeaway #2: Geode Powers Modern, Data-Driven Architectures

If the previously mentioned talks didn’t give you a sense of Geode’s fit in data-driven architectures, we also heard from a number of architects on design patterns for Geode in modern applications.

Where Does Geode Fit in Modern System Architectures?

With years of experience using Geode, Eitan Suez explained how developers and architects, like him, get started with Geode and realize that it can be used to solve a lot of problems—that it is basically a distributed hash map. He then presented an approach to using Geode as part of a CQRS (Command Query Responsibility Segregation) architecture.

Slides can be found here.

Architecting Data-Driven, Smarter Cloud Native Apps with Real-Time Decision-Making

Next, we heard from Fred Melo. He presented a compelling explanation and demo of HTAP (Hybrid Transactional / Analytical Processing) architectures, with real-time predictive analytics, featuring Apache Geode.

Slides can be found here.

Large Scale Fraud Detection Using GemFire Integrated with Greenplum

Finally, we heard about a fraud detection case study where Apache Geode was deployed as a pre-processing engine to detect events in streaming data before being loaded into a data warehouse. This included a business-logic processing layer for custom applications above the database. This particular application used the GemFire Connector to Greenplum Database.

Slides can be found here.

Takeaway #3: Geode Has a Promising Future Ahead

In another track of the Summit, many of the committers and contributors of Apache Geode convened to discuss the current and future state of Apache Geode’s feature set.  When you look at the list of capabilities below, you can see that Apache Geode as some very compelling potential features in its future.

• Spring Data GemFire API current & future
• Integration & Future Direction for Spring Cloud Data Flow & Geode
• Redis to Geode Adaptor
• Off-Heap Storage Current and Future Design
• Integration with Spark – Geode reading and writing RDDs

Many of these features are still in their conceptual phase, and the Apache Geode community is seeking people interested to help contribute to make these features better.  We finished up with a great talk by Nitin Lamba who described his journey to becoming an Apache Geode committer: Easy Ways to Become a Contributor to Apache Geode. You can also join the mailing list.

Time to Tailor Your Schedule for Geode Summit – This Weds, March 9th

We’ve just put the final touches on the schedule for Geode Summit. It’s going to be an amazing day with over 100 users and developers of Apache Geode (incubating) meeting for the very first time. We’ll be sharing expertise, networking for jobs, hearing about use cases, solution architectures, future feature designs, and conducting hands-on training.

There are two concurrent tracks running most of the day, some of which have limited space, and some of which can be attended virtually if you aren’t able to be in Palo Alto on March 9.  All the sessions, other than the hands-on lab, are being recorded so you can view them later.

If you haven’t registered yet, there’s still a little space.

There are three main types of sessions at Geode Summit:

Keynotes, Solutions, and Use Cases

This is the main track, available for all attendees, and especially suitable for technology mangers and solution architects.  Here you’ll have a chance to hear what people have done with Geode and GemFire in the past, and get some new ideas of what you can build.

Technology training

This is a 2 hour hands-on training class in the morning for developers who are new to using Apache Geode and GemFire and would like a quick introduction. Space is limited, so be sure to register and add this to your schedule if you want to participate.

Note that Participants must bring their own laptop with the following minimum configuration: Windows/Mac/Linux OS with minimum 8GB RAM Multi-core processor is highly recommended Lab environment and files will require roughly 500MB of space Have Virtual Box installed (we will deliver the environment as a VM with components installed and ready to go)

Contributor Feature Discussions

These are 30 minute deep dives into early / nascent features of Apache Geode.  An Apache Geode Committer will head a short discussion about the current capability of a feature, and then ask for ideas and use cases about how this feature could be used.  The purpose is to help inform the future direction of specific Geode capabilities.

These sessions are streamed virtually for the benefit of the Apache Geode community. You can log in here to participate: https://pivotalcommunity.adobeconnect.com/clubhouse/

Specifically these session, and their broadcast times are:

• Spring Data GemFire API current & future (1300 PST)
• Integration & Future Direction for Spring Cloud Data Flow & Geode (1330 PST)
• Redis to Geode Adaptor (1400 PST)
• Off-Heap Storage Current and Future Design (1430 PST)
• Integration with Spark – Geode reading and writing RDDs (1515 PST)
• Easy Ways to Become a Contributor to Apache Geode (1545 PST)

See you either in person, or virtually at Geode Summit!

Meet Experts and Productive Users of Apache Geode & GemFire at Geode Summit

Something amazing is happening in Palo Alto March 9. It’s the first gathering of over 100 users and developers of the GemFire and Apache Geode (incubating) in-memory data grid technology at Geode Summit.  What were previously dark arts and specialized techniques will be revealed to the open source community. Speakers from a dozen companies that have deployed or are running some of the largest production instances of the technology will share their experience and use cases. There will be hands-on training, and keynotes from industry luminaries about the future of in-memory computing and big data.

Geode was born in the fallout from Enron

The technology that became Apache Geode (incubating) was first invented trying to help a financial services startup scale up to meet new opportunity as a result of the spectacular failure of Enron.  Watch this interview with Richard Lamb, a co-founder of the technology, about genesis of GemFire by Gemstone Systems.

GemFire continued a successful growth path once Gemstone Systems was acquired by VMWare, and then spun out to become a key part of Pivotal.  Last year Pivotal proposed to The Apache Software Foundation creation of the Apache Geode (incubating) community to help foster ongoing development of the technology collaboratively with the open source community. 

An unforgettable lineup of speakers & sessions

 Geode Summit is the first time that committers and new users of Apache Geode can come together, and also meet with long standing users and developers of the commercial GemFire software.

 The morning kicks off with an opening keynote from Justin Erenkrantz, head of compute architecture at Bloomberg. Justin is also well known for his mentoring and contributions to many open source projects in The Apache Software Foundation.

 After his talk, Justin will serve as the moderator for a panel of noted industry pioneers, investors, and startup founders who are building the next generation of technologies for in-memory computing.

The amazing day continues with a track of 12 sessions by users, implementers, and developers of Apache Geode and GemFire.  There’s an optional break out in the morning for developers looking to get introductory hands-on training in how to get started with Apache Geode.

Speakers from companies such as Ampool, Capgemini, CIBC, Data Torrent, Murex, Societe Generale, Southwest Airlines, TEKSystems, zData will share their experience building solutions with Geode and GemFire.  Hear about use cases such as:

• real-time mobile apps
• streaming analytics
• fraud detection
• vehicle telematics
• financial risk computation
• adaptive logistics
• predictive analytics in manufacturing

Register today – space is limited!

We’ve already seen an outpouring of interest by experienced and new users of Apache Geode. We expect the summit to sell out.  Be sure to sign up today to reserve your spot in this amazing day.

Celebrating an Amazing 2015 for Pivotal Big Data Communities

In 2015, Pivotal released more than 6 million lines of code into open source, and we launched 4 major new open source projects:

• Apache Geode
• Apache HAWQ
• Apache MADlib
• Greenplum Database

This was a significant accomplishment by Pivotal in terms of engineering, legal, product development, and marketing efforts.

 

Since then we’ve seen quite a bit of interest in these nascent new communities in 2015.

 

2016 is our year to build them to critical mass of members with first official releases, simplifying the path to adoption and contribution, and driving awareness.

 

Here is Happy New Year message from Pivotal to the Pivotal Big Data Community.

Two Reasons Why In-Memory Data Grids Are A Must-Have For Apps at Scale

Cross posted from The Pivotal POV Blog…

Pivotal is proud to announce that Pivotal GemFire was cited as a leader in newly published the Forrester Wave™: In-Memory Data Grids, Q3 2015 report from Forrester Research. The report evaluated the respective in-memory solutions against 32 criteria in three high-level buckets of current offering, strategy, and market presence. GemFire was cited among the second-highest in the strategy category. Forrester provides the following comments about GemFire:

What to Expect from Pivotal at ApacheCon Europe 2015

Cross posted from The Pivotal POV Blog…

If anything exemplifies Pivotal’s all-in for open source direction, more than 20 sessions and events for ApacheCon Europe would be one such indicator. We are assembling a dozen open source contributors from 3 continents to help educate and inspire our fellow Apache contributors for projects we commit to such as: Apache Geode (incubating), Apache Ambari, Apache Hadoop, Apache Zeppelin (incubating), Apache Tomcat, Apache HTTP Server,Spring XD, Cloud Foundry and the Open Data Platform Initiative.

Students: “Be” The Job You Want Before You Graduate

“How to act like a baby unicorn”

It’s never been harder for a new grad to get a job in Silicon Valley, and it’s also never been easier. There are a number of forces that have converged to drastically change how new grads have to find their first job today versus how I was able to look for my first job years ago.