Nat  Kutch

Nat Kutch

1596910140

Pronto! Intuit Releases First Open Source Cassandra Cluster Manager

Just in time for the U.S. tax season’s delayed 2020 deadline, Intuit released the first framework to manage Apache Cassandra clusters. After nine years using the open source database, we made our first major contribution to the Cassandra community with DSE Pronto. Pronto is an Infrastructure as a Service automation suite used to deploy and manage DataStax Cassandra clusters in Amazon Web Services (AWS).

As its name suggests, Pronto aims to get you up and running with Cassandra much sooner.

Pronto ties together an open source suite that includes Packer, Terraform and Ansible, all built into a Docker image. The widely adopted nature of these tools also means the framework should be also easily extendible to the other two cloud giants, Google Cloud Platform and Azure.

Pronto is the result of nine years of customizations we’ve made to Cassandra — a database system that is highly reliable and scalable, but not always intuitive.

DSE Pronto Abstracts the Complexity out of Managing Cassandra

My colleagues at Intuit Ben Covi and Nancy Li developed the Pronto GitHub repository, after finding no third-party suite of similar tools with the desired configurability for self-managed clusters. It’s not easy to manage your own Cassandra cluster. Pronto solves that problem.

Pronto originated as a project with the Data Persistence Platform team, of which TurboTax is the biggest user. TurboTax is in a well-regulated industry and needs to maintain tax data for at least seven years, with hundreds of thousands of integration partners. So TurboTax is anything but simple.

We are supporting over 300,000 concurrent users actively in production in AWS, over eight clusters in production. Our largest cluster in production right now is 72 servers in each data center, or 144 across two regions. Cassandra has to process massive amounts of data, such as entitlements, tax returns, filings, user experience, and everything needed to support TurboTax.

There’s an operational learning curve with Cassandra, which is why we decided to open source the Pronto automation framework that’s already being “inner-sourced” across Intuit.

Sponsor Note

DataStax is the company behind the massively scalable, highly available, cloud native NoSQL data platform built on Apache Cassandra™️. DataStax gives users and enterprises the freedom to run data in any cloud at global scale with zero downtime and zero lock-in.

It’s actually not easy to maintain the Cassandra clusters. A lot of people don’t maintain Cassandra well and they end up in a bad state. Our automation framework is popular in Intuit and makes it easier to maintain and keep Cassandra healthy.

When we first started, we implemented Cassandra for each tax year, which meant each tax year had a new cluster. But going back seven years of seven clusters was too expensive and difficult to manage. So we consolidated four years of tax clusters into one.

When we had one year per cluster, it masked a lot of problems. After consolidating, we realized there were a lot of pauses.

I personally went through debugging and optimizing all the way from the kernel, JVM, to Cassandra level. Cassandra doesn’t remove deleted data well. We are running production tests every week to make sure services don’t degrade every time, and have to run Cassandra Garbage Collector to reclaim disk space.

#data #open source #contributed #sponsored #data analysis

What is GEEK

Buddha Community

Pronto! Intuit Releases First Open Source Cassandra Cluster Manager
Nat  Kutch

Nat Kutch

1596910140

Pronto! Intuit Releases First Open Source Cassandra Cluster Manager

Just in time for the U.S. tax season’s delayed 2020 deadline, Intuit released the first framework to manage Apache Cassandra clusters. After nine years using the open source database, we made our first major contribution to the Cassandra community with DSE Pronto. Pronto is an Infrastructure as a Service automation suite used to deploy and manage DataStax Cassandra clusters in Amazon Web Services (AWS).

As its name suggests, Pronto aims to get you up and running with Cassandra much sooner.

Pronto ties together an open source suite that includes Packer, Terraform and Ansible, all built into a Docker image. The widely adopted nature of these tools also means the framework should be also easily extendible to the other two cloud giants, Google Cloud Platform and Azure.

Pronto is the result of nine years of customizations we’ve made to Cassandra — a database system that is highly reliable and scalable, but not always intuitive.

DSE Pronto Abstracts the Complexity out of Managing Cassandra

My colleagues at Intuit Ben Covi and Nancy Li developed the Pronto GitHub repository, after finding no third-party suite of similar tools with the desired configurability for self-managed clusters. It’s not easy to manage your own Cassandra cluster. Pronto solves that problem.

Pronto originated as a project with the Data Persistence Platform team, of which TurboTax is the biggest user. TurboTax is in a well-regulated industry and needs to maintain tax data for at least seven years, with hundreds of thousands of integration partners. So TurboTax is anything but simple.

We are supporting over 300,000 concurrent users actively in production in AWS, over eight clusters in production. Our largest cluster in production right now is 72 servers in each data center, or 144 across two regions. Cassandra has to process massive amounts of data, such as entitlements, tax returns, filings, user experience, and everything needed to support TurboTax.

There’s an operational learning curve with Cassandra, which is why we decided to open source the Pronto automation framework that’s already being “inner-sourced” across Intuit.

Sponsor Note

DataStax is the company behind the massively scalable, highly available, cloud native NoSQL data platform built on Apache Cassandra™️. DataStax gives users and enterprises the freedom to run data in any cloud at global scale with zero downtime and zero lock-in.

It’s actually not easy to maintain the Cassandra clusters. A lot of people don’t maintain Cassandra well and they end up in a bad state. Our automation framework is popular in Intuit and makes it easier to maintain and keep Cassandra healthy.

When we first started, we implemented Cassandra for each tax year, which meant each tax year had a new cluster. But going back seven years of seven clusters was too expensive and difficult to manage. So we consolidated four years of tax clusters into one.

When we had one year per cluster, it masked a lot of problems. After consolidating, we realized there were a lot of pauses.

I personally went through debugging and optimizing all the way from the kernel, JVM, to Cassandra level. Cassandra doesn’t remove deleted data well. We are running production tests every week to make sure services don’t degrade every time, and have to run Cassandra Garbage Collector to reclaim disk space.

#data #open source #contributed #sponsored #data analysis

Tyrique  Littel

Tyrique Littel

1598461200

An Open-Source Book About the Open Source World

Open source today is a word that often include a lot of things, such as open knowledge (Wikimedia projects), open hardware (Arduino, Raspberry Pi), open formats (ODT/ODS/ODP) and so on.

It is a world of opportunities that can be difficult for newcomers but also for intermediates. This article will help you discover how to approach specific roles, activities or projects/communities in the best way.

Everything Started with “Coaching for OpenSource Communities 2.0”

I decided to write a book in my personal style about my experience in the last 7 to 8 years in open source. I was surprised when I reached 100 pages about various different topics.

My idea was to write something that I would like to read, so nothing that is boring or complicated, but full of real facts.

The second goal was to include my experience but also my philosophy on contributing and how I contribute daily.

Thirdly, I wanted to give a lot of hints and resources and an overall view of this open source world.

Basically, I wanted to write something different from self-help or coaching books that includes just a list of suggestions and best practices. Instead, I take real examples from real life about the OSS world.

As a contributor and developer, I prefer to have real cases to study, because best practices are useful, but we need to learn from others and this world is full of good and bad cases to discover.

In 2019, I started writing a book after Fosdem 2019 and after 2 years inside the Mozilla Reps Council. In that Fosdem edition, I had a talk “Coaching for Open Source Communities 2.0” and after the feedback at the conference and my thoughts in various roles, activities, and projects, it was time to write something.

At the end it wasn’t a manual but a book that included my experience, learnings, best practices and so on in Localization, Development, Project Maintainer, Sysadmin, Community Management, Mentor, Speaker and so on. It contains the following sections:

  • Biography - This choice isn’t for self promotion but just to understand my point of view and my story that can be inspiring for others
  • Philosophy - Not the usual description of Open Source or the 4 freedoms, but just what Open Source means and how you can help
  • How to live inside the Open Source - A discovery about communications and tools, understanding the various kind of people and the best way to talk with your community
  • How to choose a project - Starting with some questions to yourself and how to involve more people in your project
  • The activity - Open Source is based on tasks that can be divided in 2 levels: Support, Testing, Marketing, Development etc
  • How to use your time - We are busy, we have a life, a job and a family but Open Source can be time-consuming
  • Why document is important - How writing documentation can be healthy for your community and the project’s future and brand

There are also three appendices that are manuals which I wrote throughout the years and gathered and improved for this book. They are about: community management, public speaking, and mentoring.

The book ends with my point of view about the future and what we have to do to change opinions about those topics.

I wrote this book and published in October 2019, but it was only possible with the help of reviews and localizers that improved and contributed. Yes, because this book is open source and free for everyone.

I picked the GPL license because this license changed the world and my life in the best way. Using this license is just a tribute. This decision usually is not clear because after all this is a book and there are better licenses like Creative Commons.

#open-source #contributing-to-open-source #programming #software-development #development #coding #books #open-source-software

Ray  Patel

Ray Patel

1623348300

Top 8 Java Open Source Projects You Should Get Your Hands-on [2021]

Learning about Java is no easy feat. It’s a prevalent and in-demand programming language with applications in numerous sectors. We all know that if you want to learn a new skill, the best way to do so is through using it. That’s why we recommend working on projects.

So if you’re a Java student, then you’ve come to the right place as this article will help you learn about the most popular Java open source projects. This way, you’d have a firm grasp of industry trends and the programming language’s applications.

However, before we discuss its various projects, it’s crucial to examine the place where you can get those projects – GitHub. Let’s begin.

#full stack development #java open source projects #java projects #open source projects #top 8 java open source projects #java open source projects

Edison  Stark

Edison Stark

1604060760

Hacktoberfest 2020: Let’s Get Hacking

It’s October and we’re calling all programmers, designers, content writers and open-source contributors to join Hacktoberfest 2020. This is a fantastic opportunity to contribute to open-source or try your hand at something new.

For those who are new to programming or open-source, you may be wondering what is open-source or Hacktoberfest.

_Open source_refers to source code that is publicly accessible and allows anyone to inspect, modify, or learn from it. Open source projects encourage collaboration and the freedom to use the software for any purpose you wish._Hacktoberfest_is a month-long celebration of open source software run by DigitalOcean and is open to everyonein our global community.

Seven years ago, Hacktoberfest kick-started the celebration along with 676 excited participants contributing to open source projects and earning a limited-edition T-shirt. Now, hundreds of thousands of developers participate in Hacktoberfest from 150 countries.

If you want to contribute to open-source projects, but don’t know where to start, then Hacktoberfest is the perfect opportunity for you.

Hacktoberfest is a month-long celebration of open source software sponsored by Digital Ocean, Intel, and DEV.

The goal of the event is to encourage participation in the open-source community all across the globe. The challenge is quite simple: open four high-quality pull requests in October on any open source project to get some swag.

Swag you say?

If you complete valid 4prs, you stand to get a T-shirt, some stickers and a cup coaster (I got one last year, I’m not sure if they’ll be doing it this year also).

They also introduced the option to plant a tree instead of receiving a T-shirt as a reward to reduce the environmental impact.

#hacktoberfest #github #git #open-source #opensource #contributing-to-open-source #open-source-contribution #first-open-source-contribution

Houston  Sipes

Houston Sipes

1600992000

Did Google Open Sourcing Kubernetes Backfired?

Over the last few years, Kubernetes have become the de-facto standard for container orchestration and has also won the race against Docker for being the most loved platforms among developers. Released in 2014, Kubernetes has come a long way with currently being used across the entire cloudscape platforms. In fact, recent reports state that out of 109 tools to manage containers, 89% of them are leveraging Kubernetes versions.

Although inspired by Borg, Kubernetes, is an open-source project by Google, and has been donated to a vendor-neutral firm — The Cloud Native Computing Foundation. This could be attributed to Google’s vision of creating a platform that can be used by every firm of the world, including the large tech companies and can host multiple cloud platforms and data centres. The entire reason for handing over the control to CNCF is to develop the platform in the best interest of its users without vendor lock-in.

#opinions #google open source #google open source tools #google opening kubernetes #kubernetes #kubernetes platform #kubernetes tools #open source kubernetes backfired