1615647960

# NumPy for Data Science: Part 1

NumPy which stands for Numerical Python is one of the most important libraries (=packages or modules) in Python. This tutorial explains the basics of NumPy and various methods of array creation. It has been written for those who want to learn the basics and various functions of NumPy. To get the full advantage of this tutorial, you should be familiar with Python programming language and object-oriented programming (OOP) concepts.
Topics discussing
NumPy Basics: importing NumPy, Ndarray, axes, rank, attributes of ndarray class
Array Creation: array() function, 1d array, 2d array, 3d array, arange() function, ones() function, zeros() function, identity() function, eye() function, full() function, random.randn() function
Creating Vectors: row vector, column vector

#object-orientation #data-science #numpy #python #arrays

1618449987

## How To Build A Data Science Career In 2021

For this week’s data science career interview, we got in touch with Dr Suman Sanyal, Associate Professor of Computer Science and Engineering at NIIT University. In this interview, Dr Sanyal shares his insights on how universities can contribute to this highly promising sector and what aspirants can do to build a successful data science career.

With industry-linkage, technology and research-driven seamless education, NIIT University has been recognised for addressing the growing demand for data science experts worldwide with its industry-ready courses. The university has recently introduced B.Tech in Data Science course, which aims to deploy data sets models to solve real-world problems. The programme provides industry-academic synergy for the students to establish careers in data science, artificial intelligence and machine learning.

“Students with skills that are aligned to new-age technology will be of huge value. The industry today wants young, ambitious students who have the know-how on how to get things done,” Sanyal said.

#careers # #data science aspirant #data science career #data science career intervie #data science education #data science education marke #data science jobs #niit university data science

1620466520

If you accumulate data on which you base your decision-making as an organization, you should probably think about your data architecture and possible best practices.

If you accumulate data on which you base your decision-making as an organization, you most probably need to think about your data architecture and consider possible best practices. Gaining a competitive edge, remaining customer-centric to the greatest extent possible, and streamlining processes to get on-the-button outcomes can all be traced back to an organization’s capacity to build a future-ready data architecture.

In what follows, we offer a short overview of the overarching capabilities of data architecture. These include user-centricity, elasticity, robustness, and the capacity to ensure the seamless flow of data at all times. Added to these are automation enablement, plus security and data governance considerations. These points from our checklist for what we perceive to be an anticipatory analytics ecosystem.

#big data #data science #big data analytics #data analysis #data architecture #data transformation #data platform #data strategy #cloud data platform #data acquisition

1617961560

## 'Commoditization Is The Biggest Problem In Data Science Education'

The buzz around data science has sent many youngsters and professionals on an upskill/reskilling spree. Prof. Raghunathan Rengasamy, the acting head of Robert Bosch Centre for Data Science and AI, IIT Madras, believes data science knowledge will soon become a necessity.

IIT Madras has been one of India’s prestigious universities offering numerous courses in data science, machine learning, and artificial intelligence in partnership with many edtech startups. For this week’s data science career interview, Analytics India Magazine spoke to Prof. Rengasamy to understand his views on the data science education market.

With more than 15 years of experience, Prof. Rengasamy is currently heading RBCDSAI-IIT Madras and teaching at the department of chemical engineering. He has co-authored a series of review articles on condition monitoring and fault detection and diagnosis. He has also been the recipient of the Young Engineer Award for the year 2000 by the Indian National Academy of Engineering (INAE) for outstanding engineers under the age of 32.

Of late, Rengaswamy has been working on engineering applications of artificial intelligence and computational microfluidics. His research work has also led to the formation of a startup, SysEng LLC, in the US, funded through an NSF STTR grant.

#people #data science aspirants #data science course director interview #data science courses #data science education #data science education market #data science interview

1611381728

Data Science becomes an important part of today industry. It use for transforming business data into assets that help organizations improve revenue, seize business opportunities, improve customer experience, reduce costs, and more. Data science became the trending course to learn in the industries these days.

Its popularity has grown over the years, and companies have started implementing data science techniques to grow their business and increase customer satisfaction. In online Data science course you learn how Data Science deals with vast volumes of data using modern tools and techniques to find unseen patterns, derive meaningful information, and make business decisions.

Advantages of Data Science:- In today’s world, data is being generated at an alarming rate in all time lots of data is generated; from the users of social networking site, or from the calls that one makes, or the data which is being generated from different business. Because of that reason the huge amount of data the value of the field of Data Science has many advantages.

Some Of The Advantages Are Mentioned Below:-

Multiple Job Options :- Because of its high demand it provides large number of career opportunities in its various fields like Data Scientist, Data Analyst, Research Analyst, Business Analyst, Analytics Manager, Big Data Engineer, etc.

Business benefits: - By Data Science Online Course you learn how data science helps organizations knowing how and when their products sell well and that’s why the products are delivered always to the right place and right time. Faster and better decisions are taken by the organization to improve efficiency and earn higher profits.

Highly Paid jobs and career opportunities: - As Data Scientist continues working in that profile and the salaries of different position are grand. According to a Dice Salary Survey, the annual average salary of a Data Scientist \$106,000 per year as we consider data.

Hiring Benefits:- If you have skills then don’t worry this comparatively easier to sort data and look for best of candidates for an organization. Big Data and data mining have made processing and selection of CVs, aptitude tests and games easier for the recruitment group.

Also Read: How Data Science Programs Become The Reason Of Your Success

Disadvantages of Data Science: - If there are pros then cons also so here we discuss both pros and cons which make you easy to choose Data Science Course without any doubts. Let’s check some of the disadvantages of Data Science:-

Data Privacy: - As we know Data is used to increase the productivity and the revenue of industry by making game-changing business decisions. But the information or the insights obtained from the data may be misused against any organization.

Cost:- The tools used for data science and analytics can cost tons to a corporation as a number of the tools are complex and need the people to undergo a knowledge Science training to use them. Also, it’s very difficult to pick the right tools consistent with the circumstances because their selection is predicated on the proper knowledge of the tools also as their accuracy in analyzing the info and extracting information.

#data science training in noida #data science training in delhi #data science online training #data science online course #data science course #data science training

1630743562

## A Wrapper for Sembast and SQFlite to Enable Easy

FHIR_DB

This is really just a wrapper around Sembast_SQFLite - so all of the heavy lifting was done by Alex Tekartik. I highly recommend that if you have any questions about working with this package that you take a look at Sembast. He's also just a super nice guy, and even answered a question for me when I was deciding which sembast version to use. As usual, ResoCoder also has a good tutorial.

I have an interest in low-resource settings and thus a specific reason to be able to store data offline. To encourage this use, there are a number of other packages I have created based around the data format FHIR. FHIR® is the registered trademark of HL7 and is used with the permission of HL7. Use of the FHIR trademark does not constitute endorsement of this product by HL7.

## Using the Db

So, while not absolutely necessary, I highly recommend that you use some sort of interface class. This adds the benefit of more easily handling errors, plus if you change to a different database in the future, you don't have to change the rest of your app, just the interface.

I've used something like this in my projects:

``````class IFhirDb {
IFhirDb();
final ResourceDao resourceDao = ResourceDao();

Future<Either<DbFailure, Resource>> save(Resource resource) async {
Resource resultResource;
try {
resultResource = await resourceDao.save(resource);
} catch (error) {
return left(DbFailure.unableToSave(error: error.toString()));
}
return right(resultResource);
}

Future<Either<DbFailure, List<Resource>>> returnListOfSingleResourceType(
String resourceType) async {
List<Resource> resultList;
try {
resultList =
await resourceDao.getAllSortedById(resourceType: resourceType);
} catch (error) {
return left(DbFailure.unableToObtainList(error: error.toString()));
}
return right(resultList);
}

Future<Either<DbFailure, List<Resource>>> searchFunction(
String resourceType, String searchString, String reference) async {
List<Resource> resultList;
try {
resultList =
await resourceDao.searchFor(resourceType, searchString, reference);
} catch (error) {
return left(DbFailure.unableToObtainList(error: error.toString()));
}
return right(resultList);
}
}
``````

I like this because in case there's an i/o error or something, it won't crash your app. Then, you can call this interface in your app like the following:

``````final patient = Patient(
resourceType: 'Patient',
name: [HumanName(text: 'New Patient Name')],
birthDate: Date(DateTime.now()),
);

final saveResult = await IFhirDb().save(patient);
``````

This will save your newly created patient to the locally embedded database.

IMPORTANT: this database will expect that all previously created resources have an id. When you save a resource, it will check to see if that resource type has already been stored. (Each resource type is saved in it's own store in the database). It will then check if there is an ID. If there's no ID, it will create a new one for that resource (along with metadata on version number and creation time). It will save it, and return the resource. If it already has an ID, it will copy the the old version of the resource into a _history store. It will then update the metadata of the new resource and save that version into the appropriate store for that resource. If, for instance, we have a previously created patient:

``````{
"resourceType": "Patient",
"id": "fhirfli-294057507-6811107",
"meta": {
"versionId": "1",
"lastUpdated": "2020-10-16T19:41:28.054369Z"
},
"name": [
{
"given": ["New"],
"family": "Patient"
}
],
"birthDate": "2020-10-16"
}
``````

And we update the last name to 'Provider'. The above version of the patient will be kept in _history, while in the 'Patient' store in the db, we will have the updated version:

``````{
"resourceType": "Patient",
"id": "fhirfli-294057507-6811107",
"meta": {
"versionId": "2",
"lastUpdated": "2020-10-16T19:45:07.316698Z"
},
"name": [
{
"given": ["New"],
"family": "Provider"
}
],
"birthDate": "2020-10-16"
}
``````

This way we can keep track of all previous version of all resources (which is obviously important in medicine).

For most of the interactions (saving, deleting, etc), they work the way you'd expect. The only difference is search. Because Sembast is NoSQL, we can search on any of the fields in a resource. If in our interface class, we have the following function:

``````  Future<Either<DbFailure, List<Resource>>> searchFunction(
String resourceType, String searchString, String reference) async {
List<Resource> resultList;
try {
resultList =
await resourceDao.searchFor(resourceType, searchString, reference);
} catch (error) {
return left(DbFailure.unableToObtainList(error: error.toString()));
}
return right(resultList);
}
``````

You can search for all immunizations of a certain patient:

``````searchFunction(
'Immunization', 'patient.reference', 'Patient/\$patientId');
``````

This function will search through all entries in the 'Immunization' store. It will look at all 'patient.reference' fields, and return any that match 'Patient/\$patientId'.

The last thing I'll mention is that this is a password protected db, using AES-256 encryption (although it can also use Salsa20). Anytime you use the db, you have the option of using a password for encryption/decryption. Remember, if you setup the database using encryption, you will only be able to access it using that same password. When you're ready to change the password, you will need to call the update password function. If we again assume we created a change password method in our interface, it might look something like this:

``````class IFhirDb {
IFhirDb();
final ResourceDao resourceDao = ResourceDao();
...
try {
} catch (error) {
}
return right(Unit);
}
``````

You don't have to use a password, and in that case, it will save the db file as plain text. If you want to add a password later, it will encrypt it at that time.

### General Store

After using this for a while in an app, I've realized that it needs to be able to store data apart from just FHIR resources, at least on occasion. For this, I've added a second class for all versions of the database called GeneralDao. This is similar to the ResourceDao, but fewer options. So, in order to save something, it would look like this:

``````await GeneralDao().save('password', {'new':'map'});
``````

The difference between these two options is that the first one will generate a key for the map being stored, while the second will store the map using the key provided. Both will return the key after successfully storing the map.

Other functions available include:

``````// deletes everything in the general store

// delete specific entry

// returns map with that key
``````

FHIR® is a registered trademark of Health Level Seven International (HL7) and its use does not constitute an endorsement of products by HL7®

## Use this package as a library

### Depend on it

Run this command:

With Flutter:

`` \$ flutter pub add fhir_db``

This will add a line like this to your package's pubspec.yaml (and run an implicit flutter pub get):

``````dependencies:
fhir_db: ^0.4.3``````

### Import it

Now in your Dart code, you can use:

``````import 'package:fhir_db/dstu2.dart';
import 'package:fhir_db/dstu2/fhir_db.dart';
import 'package:fhir_db/dstu2/general_dao.dart';
import 'package:fhir_db/dstu2/resource_dao.dart';
import 'package:fhir_db/encrypt/aes.dart';
import 'package:fhir_db/encrypt/salsa.dart';
import 'package:fhir_db/r4.dart';
import 'package:fhir_db/r4/fhir_db.dart';
import 'package:fhir_db/r4/general_dao.dart';
import 'package:fhir_db/r4/resource_dao.dart';
import 'package:fhir_db/r5.dart';
import 'package:fhir_db/r5/fhir_db.dart';
import 'package:fhir_db/r5/general_dao.dart';
import 'package:fhir_db/r5/resource_dao.dart';
import 'package:fhir_db/stu3.dart';
import 'package:fhir_db/stu3/fhir_db.dart';
import 'package:fhir_db/stu3/general_dao.dart';
import 'package:fhir_db/stu3/resource_dao.dart'; ``````

example/lib/main.dart

``````import 'package:fhir/r4.dart';
import 'package:fhir_db/r4.dart';
import 'package:flutter/material.dart';
import 'package:test/test.dart';

Future<void> main() async {
WidgetsFlutterBinding.ensureInitialized();

final resourceDao = ResourceDao();

// await resourceDao.updatePw('newPw', null);
await resourceDao.deleteAllResources(null);

test('Playing with Passwords', () async {
final patient = Patient(id: Id('1'));

final saved = await resourceDao.save(null, patient);

await resourceDao.updatePw(null, 'newPw');
final search1 = await resourceDao.find('newPw',
resourceType: R4ResourceType.Patient, id: Id('1'));
expect(saved, search1[0]);

resourceType: R4ResourceType.Patient, id: Id('1'));
expect(saved, search2[0]);

final search3 = await resourceDao.find(null,
resourceType: R4ResourceType.Patient, id: Id('1'));
expect(saved, search3[0]);

await resourceDao.deleteAllResources(null);
});
});

final id = Id('12345');
group('Saving Things:', () {
test('Save Patient', () async {
final humanName = HumanName(family: 'Atreides', given: ['Duke']);
final patient = Patient(id: id, name: [humanName]);
final saved = await resourceDao.save(null, patient);

expect(saved.id, id);

expect((saved as Patient).name?[0], humanName);
});

test('Save Organization', () async {
final organization = Organization(id: id, name: 'FhirFli');
final saved = await resourceDao.save(null, organization);

expect(saved.id, id);

expect((saved as Organization).name, 'FhirFli');
});

test('Save Observation1', () async {
final observation1 = Observation(
id: Id('obs1'),
code: CodeableConcept(text: 'Observation #1'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save(null, observation1);

expect(saved.id, Id('obs1'));

expect((saved as Observation).code.text, 'Observation #1');
});

test('Save Observation1 Again', () async {
final observation1 = Observation(
id: Id('obs1'),
code: CodeableConcept(text: 'Observation #1 - Updated'));
final saved = await resourceDao.save(null, observation1);

expect(saved.id, Id('obs1'));

expect((saved as Observation).code.text, 'Observation #1 - Updated');

expect(saved.meta?.versionId, Id('2'));
});

test('Save Observation2', () async {
final observation2 = Observation(
id: Id('obs2'),
code: CodeableConcept(text: 'Observation #2'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save(null, observation2);

expect(saved.id, Id('obs2'));

expect((saved as Observation).code.text, 'Observation #2');
});

test('Save Observation3', () async {
final observation3 = Observation(
id: Id('obs3'),
code: CodeableConcept(text: 'Observation #3'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save(null, observation3);

expect(saved.id, Id('obs3'));

expect((saved as Observation).code.text, 'Observation #3');
});
});

group('Finding Things:', () {
test('Find 1st Patient', () async {
final search = await resourceDao.find(null,
resourceType: R4ResourceType.Patient, id: id);
final humanName = HumanName(family: 'Atreides', given: ['Duke']);

expect(search.length, 1);

expect((search[0] as Patient).name?[0], humanName);
});

test('Find 3rd Observation', () async {
final search = await resourceDao.find(null,
resourceType: R4ResourceType.Observation, id: Id('obs3'));

expect(search.length, 1);

expect(search[0].id, Id('obs3'));

expect((search[0] as Observation).code.text, 'Observation #3');
});

test('Find All Observations', () async {
final search = await resourceDao.getResourceType(
null,
resourceTypes: [R4ResourceType.Observation],
);

expect(search.length, 3);

final idList = [];
for (final obs in search) {
}

expect(idList.contains('obs1'), true);

expect(idList.contains('obs2'), true);

expect(idList.contains('obs3'), true);
});

test('Find All (non-historical) Resources', () async {
final search = await resourceDao.getAll(null);

expect(search.length, 5);
final patList = search.toList();
final orgList = search.toList();
final obsList = search.toList();
patList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Patient);
orgList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Organization);
obsList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Observation);

expect(patList.length, 1);

expect(orgList.length, 1);

expect(obsList.length, 3);
});
});

group('Deleting Things:', () {
test('Delete 2nd Observation', () async {
await resourceDao.delete(
null, null, R4ResourceType.Observation, Id('obs2'), null, null);

final search = await resourceDao.getResourceType(
null,
resourceTypes: [R4ResourceType.Observation],
);

expect(search.length, 2);

final idList = [];
for (final obs in search) {
}

expect(idList.contains('obs1'), true);

expect(idList.contains('obs2'), false);

expect(idList.contains('obs3'), true);
});

test('Delete All Observations', () async {
await resourceDao.deleteSingleType(null,
resourceType: R4ResourceType.Observation);

final search = await resourceDao.getAll(null);

expect(search.length, 2);

final patList = search.toList();
final orgList = search.toList();
patList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Patient);
orgList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Organization);

expect(patList.length, 1);

expect(patList.length, 1);
});

test('Delete All Resources', () async {
await resourceDao.deleteAllResources(null);

final search = await resourceDao.getAll(null);

expect(search.length, 0);
});
});

group('Password - Saving Things:', () {
test('Save Patient', () async {
await resourceDao.updatePw(null, 'newPw');
final humanName = HumanName(family: 'Atreides', given: ['Duke']);
final patient = Patient(id: id, name: [humanName]);
final saved = await resourceDao.save('newPw', patient);

expect(saved.id, id);

expect((saved as Patient).name?[0], humanName);
});

test('Save Organization', () async {
final organization = Organization(id: id, name: 'FhirFli');
final saved = await resourceDao.save('newPw', organization);

expect(saved.id, id);

expect((saved as Organization).name, 'FhirFli');
});

test('Save Observation1', () async {
final observation1 = Observation(
id: Id('obs1'),
code: CodeableConcept(text: 'Observation #1'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save('newPw', observation1);

expect(saved.id, Id('obs1'));

expect((saved as Observation).code.text, 'Observation #1');
});

test('Save Observation1 Again', () async {
final observation1 = Observation(
id: Id('obs1'),
code: CodeableConcept(text: 'Observation #1 - Updated'));
final saved = await resourceDao.save('newPw', observation1);

expect(saved.id, Id('obs1'));

expect((saved as Observation).code.text, 'Observation #1 - Updated');

expect(saved.meta?.versionId, Id('2'));
});

test('Save Observation2', () async {
final observation2 = Observation(
id: Id('obs2'),
code: CodeableConcept(text: 'Observation #2'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save('newPw', observation2);

expect(saved.id, Id('obs2'));

expect((saved as Observation).code.text, 'Observation #2');
});

test('Save Observation3', () async {
final observation3 = Observation(
id: Id('obs3'),
code: CodeableConcept(text: 'Observation #3'),
effectiveDateTime: FhirDateTime(DateTime(1981, 09, 18)),
);
final saved = await resourceDao.save('newPw', observation3);

expect(saved.id, Id('obs3'));

expect((saved as Observation).code.text, 'Observation #3');
});
});

group('Password - Finding Things:', () {
test('Find 1st Patient', () async {
final search = await resourceDao.find('newPw',
resourceType: R4ResourceType.Patient, id: id);
final humanName = HumanName(family: 'Atreides', given: ['Duke']);

expect(search.length, 1);

expect((search[0] as Patient).name?[0], humanName);
});

test('Find 3rd Observation', () async {
final search = await resourceDao.find('newPw',
resourceType: R4ResourceType.Observation, id: Id('obs3'));

expect(search.length, 1);

expect(search[0].id, Id('obs3'));

expect((search[0] as Observation).code.text, 'Observation #3');
});

test('Find All Observations', () async {
final search = await resourceDao.getResourceType(
'newPw',
resourceTypes: [R4ResourceType.Observation],
);

expect(search.length, 3);

final idList = [];
for (final obs in search) {
}

expect(idList.contains('obs1'), true);

expect(idList.contains('obs2'), true);

expect(idList.contains('obs3'), true);
});

test('Find All (non-historical) Resources', () async {
final search = await resourceDao.getAll('newPw');

expect(search.length, 5);
final patList = search.toList();
final orgList = search.toList();
final obsList = search.toList();
patList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Patient);
orgList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Organization);
obsList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Observation);

expect(patList.length, 1);

expect(orgList.length, 1);

expect(obsList.length, 3);
});
});

group('Password - Deleting Things:', () {
test('Delete 2nd Observation', () async {
await resourceDao.delete(
'newPw', null, R4ResourceType.Observation, Id('obs2'), null, null);

final search = await resourceDao.getResourceType(
'newPw',
resourceTypes: [R4ResourceType.Observation],
);

expect(search.length, 2);

final idList = [];
for (final obs in search) {
}

expect(idList.contains('obs1'), true);

expect(idList.contains('obs2'), false);

expect(idList.contains('obs3'), true);
});

test('Delete All Observations', () async {
await resourceDao.deleteSingleType('newPw',
resourceType: R4ResourceType.Observation);

final search = await resourceDao.getAll('newPw');

expect(search.length, 2);

final patList = search.toList();
final orgList = search.toList();
patList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Patient);
orgList.retainWhere(
(resource) => resource.resourceType == R4ResourceType.Organization);

expect(patList.length, 1);

expect(patList.length, 1);
});

test('Delete All Resources', () async {
await resourceDao.deleteAllResources('newPw');

final search = await resourceDao.getAll('newPw');

expect(search.length, 0);

await resourceDao.updatePw('newPw', null);
});
});
} ``````