Massive amounts of data are causing challenges for businesses, making it tough for even data scientists to deal with it. Here are the books every data scientist should read in 2022 for a deeper outlook.
Know more: https://albertchristopher.substack.com/p/pivotal-classics-that-data-scientists-must-consider-in-2022-2680de0a045c
February 15, 2022 marked a significant milestone in Atlassian’s Server EOL (End Of Life) roadmap. This was not the final step. We still have two major milestones ahead of us: end of new app sales in Feb 2023, and end of support in Feb 2024. In simpler words, businesses still have enough time to migrate their Jira Server to one of the two available products – Atlassian Cloud or Atlassian DC. But the clock is ticking.
If we were to go by Atlassian numbers, 95% of their new customers choose cloud.
“About 80% of Fortune 500 companies have an Atlassian Cloud license. More than 90% of new customers choose cloud first.” – Daniel Scott, Product Marketing Director, Tempo
So that’s settled, right? We are migrating from Server to Cloud? And what about the solution fewer people talk about yet many users rely on – Jira DC?
Both are viable options and your choice will depend greatly on the needs of your business, your available resources, and operational processes.
Let’s start by taking a look at the functionality offered by Atlassian Cloud and Atlassian DC.
|Feature||Atlassian Cloud||Atlassian Data Center|
|Product Plans||Multiple plans||One plan|
|Billing||Monthly and annual||Annual only|
|Pricing model||Per user or tiered||Tiered only|
|Support||Varying support levels depending on your plan: Enterprise support coverage is equivalent to Atlassian’s Data Center Premier Support offering||Varying support levels depending on the package: Priority Support or Premier Support (purchased separately)|
|Total Cost of Ownership||TCO includes your subscription fee, plus product administration time||TCO includes your subscription fee and product administration time, plus: costs related to infrastructure provisioning or IaaS fees (for example, AWS costs) planned downtime time and resources needed for software upgrades|
|Data encryption services|
|Data residency services|
|Audit logging||Organization-level audit logging available via Atlassian Access (Jira Software, Confluence) |
Product-level audit logs (Jira Software, Confluence)
|Advanced audit logging|
|Device security||Mobile device management support (Jira Software, Confluence, Jira Service Management)|
Mobile application management (currently on the roadmap)
|Mobile device management support (Jira Software, Confluence, Jira Service Management)|
|Data Storage limits||2 GB (Free)|
250 GB (Standard)
Unlimited storage (Premium and Enterprise)
|Performance||Continuous performance updates to improve load times, search responsiveness, and attachments|
Cloud infrastructure hosted in six geographic regions to reduce latency
|Rate limitingCDN supports Smart mirrors and mirror farms (Bitbucket)|
|Backup and data disaster recovery||Jira leverages multiple geographically diverse data centers, has a comprehensive backup program, and gains assurance by regularly testing their disaster recovery and business continuity plans. |
Backups are generated daily and retained for 30 days to allow for point-in-time data restoration
|Containerization and orchestration||Docker images|
Kubernetes support (on the roadmap for now)
|Change management and upgrades||Atlassian automatically handles software and security upgrades for you Sandbox instance to test changes (Premium and Enterprise) |
Release track options for Premium and Enterprise (Jira Software, Jira Service Management, Confluence)
|Direct access to the database||No direct access to change the database structure, file system, or other server infrastructure|
Extensive REST APIs for programmatic data access
|Direct database access|
|Insights and reporting||Organization and admin insights to track adoption of Atlassian products, and evaluate the security of your organization.||Data Pipeline for advanced insightsConfluence analytics|
When talking about pros and cons, there’s always a chance that a competitive advantage for some is a dealbreaker for others. That’s why I decided to talk about pros and cons in matching pairs.
Pro: Scalability is one of the primary reasons businesses are choosing Jira Cloud. DC is technically also scalable, but you’ll need to scale on your own whereas the cloud version allows for the infrastructure to scale with your business.
Con: Despite the cloud’s ability to grow with your business, there is still a user limit of 35k users. In addition to that, the costs will grow alongside your needs. New users, licenses, storage, and computing power – all come at an additional cost. So, when your organization reaches a certain size, migrating to Jira DC becomes more cost-efficient.
Pro: Jira takes care of maintenance and support for you.
Con: Your business can suffer from unpredicted downtime. And there are certain security risks.
Pro: Extra bells and whistles:
Con: Most of these features are locked behind a paywall and are only available to either Premium and Enterprise or only Enterprise licenses (either fully or through addition of functionality. For example, Release tracks are only available to Enterprise customers.) In addition, the costs will grow as you scale the offering to fit your growing needs.
I’ll be taking the same approach to talking about the pros and cons as I did when writing about Atlassian Cloud. Pros and cons are paired.
Pro: Hosting your own system means you can scale horizontally and vertically through additional hardware. Extension of your systems is seamless, and there is no downtime (if you do everything correctly). Lastly, you don’t have to worry about the user limit – there is none.
Con: While having more control over your systems is great, it implies a dedicated staff of engineers, additional expenses on software licensing, hardware, and physical space. Moreover, seamless extension and 0% downtime are entirely on you.
Pro: Atlassian has updated the DC offering with native bundled applications such as Advanced Roadmaps, team calendars and analytics for confluence, insight asset management, and insight discovery in Jira Service Management DC.
Con: Atlassian has updated their pricing to reflect these changes. And you are still getting fewer “bells and whistles” than Jira Cloud users (as we can see from the feature comparison).
Pro: You are technically safer as the system is supported on your hardware by your specialists. Any and all Jira server issues, poor updates, and downtime are simply not your concern.
Con: Atlassian offers excellent security options: data encryption in transit and rest, to mobile app management, to audit offerings and API token controls. In their absence, your team company has to dedicate additional resources to security.
Pro: Additional benefits from Atlassian, such as the Priority Support bundle (all DC subscriptions have this option), and the Data center loyalty discount (more on that in the pricing section.)
Talking about pricing of SaaS products is always a challenge as there are always multiple tiers and various pay-as-you go features. Barebones Jira Cloud, for instance, is completely free of charge, yet there are a series of serious limitations.
Standard Jira Cloud will cost you an average of $7.50 per user per month while premium cranks that price up to $14.50. The Enterprise plan is billed annually and the cost is determined on a case-by-case basis. You can see the full comparison of Jira Cloud plans here. And you can use this online calculator to learn the cost of ownership in your particular case.
|50 Users||Standard (Monthly/Annually)||Premium (Monthly/Annually)|
|Jira Software||$387.50 / $3,900||$762.50 / $7,650|
|Jira Work Management||$250 / $2,500|
|Jira Service Management||$866.25 / $8,650||$2,138.25 / $21,500|
|Confluence||$287.50 / $2,900||$550 / $5,500|
|100 Users||Standard (Monthly/Annually)||Premium (Monthly/Annually)|
|Jira Software||$775 / $7,750||$1,525 / $15,250|
|Jira Work Management||$500 / $5,000|
|Jira Service Management||$1,653.75 / $16,550||$4,185.75 / $42,000|
|Confluence||$575 / $5,750||$1,100 / $11,000|
|500 Users||Standard (Monthly/Annually)||Premium (Monthly/Annually)|
|Jira Software||$3,140 / $31,500||$5,107.50 / $51,000|
|Jira Work Management||$1,850 / $18,500|
|Jira Service Management||$4,541.25 / $45,400||$11,693.25 / $117,000|
|Confluence||$2,060 / $20,500||$3,780 / $37,800|
Please note that these prices were calculated without any apps included.
Jira Data Center starts at $42,000 per year and the plan includes up to 500 users. If you are a new client and are not eligible for any discounts*, here’s a chart that should give you an idea as to the cost of ownership of Jira DC. You can find more information regarding your specific case here.
|Users||Commercial Annual Plan||Academic Annual Plan|
|1-500||USD 42,000||USD 21,000|
|501-1000||USD 72,000||USD 36,000|
|1001-2000||USD 120,000||USD 60,000|
|Confluence for Data Center|
|1-500||USD 27,000||USD 13,500|
|501-1000||USD 48,000||USD 24,000|
|1001-2000||USD 84,000||USD 42,000|
|Bitbucket for Data Center|
|1-25||USD 2,300||USD 1,150|
|26-50||USD 4,200||USD 2,100|
|51-100||USD 7,600||USD 3,800|
|Jira Service Management for Data Center|
|1-50||USD 17,200||USD 8,600|
|51-100||USD 28,600||USD 14,300|
|101-250||USD 51,500||USD 25,750|
Originally, there were several migration methods: Jira Cloud Migration Assistant, Jira Cloud Site Import, and there was an option to migrate via CSV export (though Jira actively discourages you from using this method). However, Jira’s team has focused their efforts on improving the Migration Assistant and have chosen to discontinue Cloud Site Import support.
Thanks to the broadened functionality of the assistant, it is now the only go-to method for migration with just one exception. If you are migrating over 1000 users and you absolutely need to migrate advanced roadmaps – you’ll need to rely on Site Import. At least for now, as Jira is actively working on implementing this feature in their assistant.
Here’s a quick comparison of the options and their limitations.
|Cloud Migration Assistant||App migration|
Existing data on a Cloud Site is not overwritten
You choose the projects, users, and groups you want to migrate
Jira Service Management customer account migration
Better UI to guide you through the migration
Potential migration errors are displayed in advance
Migration can be done in phases reducing the downtime
Pre- and post-migration reports
|You must be on a supported self-managed version of Jira|
|Site Export||Can migrate Advanced Roadmaps||App data is not migrated|
Migration overrides existing data on the Cloud site
Separate user importUsers from external directories are not migrated
No choice of data you want or don’t want migrated
There’s a need to split attachments into up to 5GB chunks
Higher risks of downtime due to the “all or nothing” approach
You must be on a supported self-managed version of Jira
Pro tip: If you have a large base of users (above 2000), migrate them before you migrate projects and spaces. This way, you will not disrupt the workflow as users are still working on Server and the latter migration of data will take less time.
Now that we have settled on one particular offering based on available pricing models as well as the pros and the cons that matter the most to your organization, let’s talk about the “how”.
How does one migrate from Jira Server to Jira Cloud?
Jira’s Cloud Migration Assistant is a handy tool. It will automatically review your data for common errors. But it is incapable of doing all of the work for you. That’s why we – and Atlassian for that matter – recommend creating a pre-migration checklist.
Smart Checklist will help you craft an actionable, context-rich checklist directly inside a Jira ticket. This way, none of the tasks will be missed, lost, or abandoned.
Below is an example of how your migration checklist will look like in Jira.
Feel free to copy the code and paste it into your Smart Checklist editor and you’ll have the checklist at the ready.
# Create a user migration plan #must > Please keep in mind that Jira Cloud Migration Assistant migrates all users and groups as well as users and groups related to selected projects - Sync your user base - Verify synchronization - External users sync verification - Active external directory verification ## Check your Jira Server version #must - Verify via user interface or Support Zip Product Version Verification > Jira Migration Assistant will not work unless Jira is running on a supported version ## Fix any duplicate email addresses #must - Verify using SQL > Duplicate email addresses are not supported by Jira Cloud and therefore can't be migrated with the Jira Cloud Migration Assistant. To avoid errors, you should find and fix any duplicate email addresses before migration. If user information is managed in an LDAP Server, you will need to update emails there and sync with Jira before the migration. If user information is managed locally, you can fix them through the Jira Server or Data Center user interface. ## Make sure you have the necessary permissions #must - System Admin global permissions on the Server instance - Exists in the target Cloud site - Site Administrator Permission in the cloud ## Check for conflicts with group names #must - Make sure that the groups in your Cloud Site don't have the same names as groups in Server > Unless you are actively trying to merge them - Delete or update add-on users so not to cause migration issues - Verify via SQL ## Update firewall allowance rules #must - None of the domains should be blocked by firewall or proxy ## Find a way to migrate apps #must - Contact app vendors ## Check public access settings #must - Projects - Filters - Filters - Boards - Dashboards ## Review server setup #mst - at least 4gb Heap Allocation - Open Files limit review - Verify via support zip ## Check Server timezone #must for merging Cloud sites - Switch to UTC is using any other timezone > Add a system flag to the Jira Server instance -Duser.timezone=UTC as outlined in this article about updating documentation to include timezone details. ## Fix any duplicate shared configuration ## Storage limits ## Prepare the server instance - Check data status - All fields have value and are not null -Any archived projects you wish to migrate are activated ## Prepare your cloud site - Same Jira products enabled - Same language - User migration strategy ## Data backup - Backup Jira Server site - Backup Cloud site ## Run a test migration - Done ## Notify Jira support - Get in touch with Jira migration support
On the one hand, having all of your Jira products on a server may seem like a backup in and of itself. On the other hand, there are data migration best practices we should follow even if it’s just a precaution. No one has ever felt sorry for their data being too safe.
In addition, there are certain types of migration errors that can be resolved much faster with having a backup at hand.
Jira Cloud Migration Assistant is a free add-on Atlassian recommends using when migrating to the cloud. It accesses and evaluates your apps and helps migrate multiple projects.
Overall, the migration assistant offers a more stable and reliable migration experience. It automatically checks for certain errors. It makes sure all users have unique and valid emails, and makes sure that none of the project names and keys conflict with one another.
This is a step-by-step guide for importing your Jira Server data backup file into Jira Cloud.
Before we can proceed with the migration process, please make sure you meet the following prerequisites:
Once you are certain you are ready to migrate your Jira Server to Jira Data Center, you can proceed with an installation that’s much simpler than one would expect.
That’s it. You are all set. Well, unless your organization has specific needs such as continuous uptime, performance under heavy loads, and scalability, in which case you will need to set up a server cluster. You can find out more about setting up server clusters in this guide.
If you accumulate data on which you base your decision-making as an organization, you should probably think about your data architecture and possible best practices.
If you accumulate data on which you base your decision-making as an organization, you most probably need to think about your data architecture and consider possible best practices. Gaining a competitive edge, remaining customer-centric to the greatest extent possible, and streamlining processes to get on-the-button outcomes can all be traced back to an organization’s capacity to build a future-ready data architecture.
In what follows, we offer a short overview of the overarching capabilities of data architecture. These include user-centricity, elasticity, robustness, and the capacity to ensure the seamless flow of data at all times. Added to these are automation enablement, plus security and data governance considerations. These points from our checklist for what we perceive to be an anticipatory analytics ecosystem.
#big data #data science #big data analytics #data analysis #data architecture #data transformation #data platform #data strategy #cloud data platform #data acquisition
Our latest survey report suggests that as the overall Data Science and Analytics market evolves to adapt to the constantly changing economic and business environments, data scientists and AI practitioners should be aware of the skills and tools that the broader community is working on. A good grip in these skills will further help data science enthusiasts to get the best jobs that various industries in their data science functions are offering.
In this article, we list down 50 latest job openings in data science that opened just last week.
(The jobs are sorted according to the years of experience r
Skills Required: Real-time anomaly detection solutions, NLP, text analytics, log analysis, cloud migration, AI planning, etc.
Skills Required: Data mining experience in Python, R, H2O and/or SAS, cross-functional, highly complex data science projects, SQL or SQL-like tools, among others.
Skills Required: Data modelling, database architecture, database design, database programming such as SQL, Python, etc., forecasting algorithms, cloud platforms, designing and developing ETL and ELT processes, etc.
Skills Required: SQL and querying relational databases, statistical programming language (SAS, R, Python), data visualisation tool (Tableau, Qlikview), project management, etc.
**Location: **Bibinagar, Telangana
Skills Required: Data science frameworks Jupyter notebook, AWS Sagemaker, querying databases and using statistical computer languages: R, Python, SLQ, statistical and data mining techniques, distributed data/computing tools such as Map/Reduce, Flume, Drill, Hadoop, Hive, Spark, Gurobi, MySQL, among others.
#careers #data science #data science career #data science jobs #data science news #data scientist #data scientists #data scientists india
The opportunities big data offers also come with very real challenges that many organizations are facing today. Often, it’s finding the most cost-effective, scalable way to store and process boundless volumes of data in multiple formats that come from a growing number of sources. Then organizations need the analytical capabilities and flexibility to turn this data into insights that can meet their specific business objectives.
This Refcard dives into how a data lake helps tackle these challenges at both ends — from its enhanced architecture that’s designed for efficient data ingestion, storage, and management to its advanced analytics functionality and performance flexibility. You’ll also explore key benefits and common use cases.
As technology continues to evolve with new data sources, such as IoT sensors and social media churning out large volumes of data, there has never been a better time to discuss the possibilities and challenges of managing such data for varying analytical insights. In this Refcard, we dig deep into how data lakes solve the problem of storing and processing enormous amounts of data. While doing so, we also explore the benefits of data lakes, their use cases, and how they differ from data warehouses (DWHs).
This is a preview of the Getting Started With Data Lakes Refcard. To read the entire Refcard, please download the PDF from the link above.
#big data #data analytics #data analysis #business analytics #data warehouse #data storage #data lake #data lake architecture #data lake governance #data lake management
Around once a month, I get emailed by a student of some type asking how to get into Data Science, I’ve answered it enough that I decided to write it out here so I can link people to it. So if you’re one of those students, welcome!
I’ll segment this into basic advice, which can be found quite easily if you just google ‘how to get into data science’ and advice that is less common, but advice that I’ve found very useful over the years. I’ll start with the latter, and move on to basic advice. Obviously take this with a grain of salt as all advice comes with a bit of survivorship bias.
#big data & cloud #data science #data scientist #statistics #aspiring data scientist #advice for aspiring data scientists