Semgrep: Lightweight Static analysis for Many Languages

Semgrep

Code scanning at ludicrous speed. 
Find bugs and reachable dependency vulnerabilities in code. 
Enforce your code standards on every commit.

Semgrep is a fast, open-source, static analysis engine for finding bugs, detecting vulnerabilities in third-party dependencies, and enforcing code standards. Get started →.

Semgrep analyzes code locally on your computer or in your build environment: code is never uploaded.

Its rules look like the code you already write; no abstract syntax trees, regex wrestling, or painful DSLs. Here's a quick rule for finding Python print() statements. Run it online in Semgrep’s Playground by clicking the image:

The Semgrep ecosystem includes:

• Semgrep - The open-source command line tool at the heart of everything (this project).
• Semgrep Supply Chain - high-signal dependency scanner that detects reachable vulnerabilities in third-party libraries and functions across the SDLC.
• Semgrep App - Deploy, manage, and monitor Semgrep and Semgrep Supply Chain at scale with free and paid tiers. Integrates with CI providers such as GitHub, GitLab, CircleCI, and more.

and:

• Semgrep Playground - An online interactive tool for writing and sharing rules.
• Semgrep Registry - 2,000+ community-driven rules covering security, correctness, and dependency vulnerabilities.

Join hundreds of thousands of other developers and security engineers already using Semgrep at companies like GitLab, Dropbox, Slack, Figma, Shopify, HashiCorp, Snowflake, and Trail of Bits.

Semgrep is developed and commercially supported by r2c, a software security company.

Language support

General availability

C# · Go · Java · JavaScript · JSX · JSON · PHP · Python · Ruby · Scala · TypeScript · TSX
 

Beta & experimental

See supported languages for the complete list.

Getting started

To install Semgrep use Homebrew or pip, or run without installation via Docker:

# For macOS
$ brew install semgrep

# For Ubuntu/WSL/Linux/macOS
$ python3 -m pip install semgrep

# To try Semgrep without installation run via Docker
$ docker run --rm -v "${PWD}:/src" returntocorp/semgrep semgrep

Once installed, Semgrep can run with single rules or entire rulesets. Visit Docs > Running rules to learn more or try the following:

# Check for Python == where the left and right hand sides are the same (often a bug)
$ semgrep -e '$X == $X' --lang=py path/to/src

# Fetch rules automatically by setting the `--config auto` flag.
# This will fetch rules relevant to your project from Semgrep Registry.
# Your source code is not uploaded.
$ semgrep --config auto

To run Semgrep Supply Chain, contact the Semgrep team. Visit the full documentation to learn more.

Rule examples

Visit Docs > Rule examples for use cases and ideas.

Use case	Semgrep rule
Ban dangerous APIs	Prevent use of exec
Search routes and authentication	Extract Spring routes
Enforce the use secure defaults	Securely set Flask cookies
Tainted data flowing into sinks	ExpressJS dataflow into sandbox.run
Enforce project best-practices	Use assertEqual for == checks, Always check subprocess calls
Codify project-specific knowledge	Verify transactions before making them
Audit security hotspots	Finding XSS in Apache Airflow, Hardcoded credentials
Audit configuration files	Find S3 ARN uses
Migrate from deprecated APIs	DES is deprecated, Deprecated Flask APIs, Deprecated Bokeh APIs
Apply automatic fixes	Use listenAndServeTLS

Extensions

Visit Docs > Extensions to learn about using Semgrep in your editor or pre-commit. When integrated into CI and configured to scan pull requests, Semgrep will only report issues introduced by that pull request; this lets you start using Semgrep without fixing or ignoring pre-existing issues!

Documentation

Browse the full Semgrep documentation on the website. If you’re new to Semgrep, check out Docs > Getting started or the interactive tutorial.

Metrics

Using remote configuration from the Registry (like --config=p/ci) reports pseudonymous rule metrics to semgrep.dev.

Using configs from local files (like --config=xyz.yml) does not enable metrics.

To disable Registry rule metrics, use --metrics=off.

The Semgrep privacy policy describes the principles that guide data-collection decisions and the breakdown of the data that are and are not collected when the metrics are enabled.

More

• Frequently asked questions (FAQs)
• Contributing
• Build instructions for developers
• Ask questions in the r2c Community Slack
• CLI reference and exit codes
• Semgrep YouTube channel
• License (LGPL-2.1)

Upgrading

To upgrade, run the command below associated with how you installed Semgrep:

# Using Homebrew
$ brew upgrade semgrep

# Using pip
$ python3 -m pip install --upgrade semgrep

# Using Docker
$ docker pull returntocorp/semgrep:latest

Download Details:

Author: Returntocorp
Source Code: https://github.com/returntocorp/semgrep 
License: View license

#typescript #javascript #ruby #python #c 

What is GEEK

Buddha Community

Static Code Analysis: What It Is? How to Use It?

Static code analysis refers to the technique of approximating the runtime behavior of a program. In other words, it is the process of predicting the output of a program without actually executing it.

Lately, however, the term “Static Code Analysis” is more commonly used to refer to one of the applications of this technique rather than the technique itself — program comprehension — understanding the program and detecting issues in it (anything from syntax errors to type mismatches, performance hogs likely bugs, security loopholes, etc.). This is the usage we’d be referring to throughout this post.

“The refinement of techniques for the prompt discovery of error serves as well as any other as a hallmark of what we mean by science.”

• J. Robert Oppenheimer

Outline

We cover a lot of ground in this post. The aim is to build an understanding of static code analysis and to equip you with the basic theory, and the right tools so that you can write analyzers on your own.

We start our journey with laying down the essential parts of the pipeline which a compiler follows to understand what a piece of code does. We learn where to tap points in this pipeline to plug in our analyzers and extract meaningful information. In the latter half, we get our feet wet, and write four such static analyzers, completely from scratch, in Python.

Note that although the ideas here are discussed in light of Python, static code analyzers across all programming languages are carved out along similar lines. We chose Python because of the availability of an easy to use ast module, and wide adoption of the language itself.

How does it all work?

Before a computer can finally “understand” and execute a piece of code, it goes through a series of complicated transformations:

As you can see in the diagram (go ahead, zoom it!), the static analyzers feed on the output of these stages. To be able to better understand the static analysis techniques, let’s look at each of these steps in some more detail:

Scanning

The first thing that a compiler does when trying to understand a piece of code is to break it down into smaller chunks, also known as tokens. Tokens are akin to what words are in a language.

A token might consist of either a single character, like (, or literals (like integers, strings, e.g., 7, Bob, etc.), or reserved keywords of that language (e.g, def in Python). Characters which do not contribute towards the semantics of a program, like trailing whitespace, comments, etc. are often discarded by the scanner.

Python provides the tokenize module in its standard library to let you play around with tokens:

Python

1

import io

2

import tokenize

3

4

code = b"color = input('Enter your favourite color: ')"

5

6

for token in tokenize.tokenize(io.BytesIO(code).readline):

7

    print(token)

Python

1

TokenInfo(type=62 (ENCODING),  string='utf-8')

2

TokenInfo(type=1  (NAME),      string='color')

3

TokenInfo(type=54 (OP),        string='=')

4

TokenInfo(type=1  (NAME),      string='input')

5

TokenInfo(type=54 (OP),        string='(')

6

TokenInfo(type=3  (STRING),    string="'Enter your favourite color: '")

7

TokenInfo(type=54 (OP),        string=')')

8

TokenInfo(type=4  (NEWLINE),   string='')

9

TokenInfo(type=0  (ENDMARKER), string='')

(Note that for the sake of readability, I’ve omitted a few columns from the result above — metadata like starting index, ending index, a copy of the line on which a token occurs, etc.)

#code quality #code review #static analysis #static code analysis #code analysis #static analysis tools #code review tips #static code analyzer #static code analysis tool #static analyzer

Advantage of C Language Certification Online Training in 2020

C language is a procedural programming language. C language is the general purpose and object oriented programming language. C language is mainly used for developing different types of operating systems and other programming languages. C language is basically run in hardware and operating systems. C language is used many software applications such as internet browser, MYSQL and Microsoft Office.
**
Advantage of doing C Language Training in 2020 are:**

1. Popular Programming language: The main Advantage of doing C language training in 2020 is popular programming language. C programming language is used and applied worldwide. C language is adaptable and flexible in nature. C language is important for different programmers. The basic languages that are used in C language is Java, C++, PHP, Python, Perl, JavaScript, Rust and C- shell.

2. Basic language of all advanced languages: The another main Advantage of doing C language training in 2020 is basic language of all advanced languages. C language is an object oriented language. For learning, other languages, you have to master in C language.

3. Understand the computer theories: The another main Advantage of doing C language training in 2020 is understand the computer theories. The theories such as Computer Networks, Computer Architecture and Operating Systems are based on C programming language.

4. Fast in execution time: The another main Advantage of doing C language training in 2020 is fast in execution time. C language is to requires small run time and fast in execution time. The programs are written in C language are faster than the other programming language.

5. Used by long term: The another main Advantage of doing C language training in 2020 is used by long term. The C language is not learning in the short span of time. It takes time and energy for becoming career in C language. C language is the only language that used by decades of time. C language is that exists for the longest period of time in computer programming history.

6. Rich Function Library: The another main Advantage of doing C language training in 2020 is rich function library. C language has rich function of libraries as compared to other programming languages. The libraries help to build the analytical skills.

7. Great degree of portability: The another main Advantage of doing C language training in 2020 is great degree of portability. C is a portable assemble language. It has a great degree of portability as compilers and interpreters of other programming languages are implemented in C language.
   The demand of C language is high in IT sector and increasing rapidly.

C Language Online Training is for individuals and professionals.
C Language Online Training helps to develop an application, build operating systems, games and applications, work on the accessibility of files and memory and many more.

C Language Online Course is providing the depth knowledge of functional and logical part, develop an application, work on memory management, understanding of line arguments, compiling, running and debugging of C programs.

Is C Language Training Worth Learning for You! and is providing the basic understanding of create C applications, apply the real time programming, write high quality code, computer programming, C functions, variables, datatypes, operators, loops, statements, groups, arrays, strings, etc.

The companies which are using C language are Amazon, Martin, Apple, Samsung, Google, Oracle, Nokia, IBM, Intel, Novell, Microsoft, Facebook, Bloomberg, VM Ware, etc.
C language is used in different domains like banking, IT, Insurance, Education, Gaming, Networking, Firmware, Telecommunication, Graphics, Management, Embedded, Application Development, Driver level Development, Banking, etc.

The job opportunities after completing the C Language Online certificationAre Data Scientists, Back End Developer, Embedded Developer, C Analyst, Software Developer, Junior Programmer, Database Developer, Embedded Engineer, Programming Architect, Game Programmer, Quality Analyst, Senior Programmer, Full Stack Developer, DevOps Specialist, Front End Web Developer, App Developer, Java Software Engineer, Software Developer and many more.

#c language online training #c language online course #c language certification online #c language certification #c language certification course #c language certification training

Effective Code Reviews: A Primer

Peer code reviews as a process have increasingly been adopted by engineering teams around the world. And for good reason — code reviews have been proven to improve software quality and save developers’ time in the long run. A lot has been written about how code reviews help engineering teams by leading software engineering practitioners. My favorite is this quote by Karl Wiegers, author of the seminal paper on this topic, Humanizing Peer Reviews:

Peer review – an activity in which people other than the author of a software deliverable examine it for defects and improvement opportunities – is one of the most powerful software quality tools available. Peer review methods include inspections, walkthroughs, peer deskchecks, and other similar activities. After experiencing the benefits of peer reviews for nearly fifteen years, I would never work in a team that did not perform them.

It is worth the time and effort to put together a code review strategy and consistently follow it in the team. In essence, this has a two-pronged benefit: more pair of eyes looking at the code decreases the chances of bugs and bad design patterns entering your codebase, and embracing the process fosters knowledge sharing and positive collaboration culture in the team.

Here are 6 tips to ensure effective peer reviews in your team.

1. Keep the Changes Small and Focused

Code reviews require developers to look at someone else’s code, most of which is completely new most of the times. Too many lines of code to review at once requires a huge amount of cognitive effort, and the quality of review diminishes as the size of changes increases. While there’s no golden number of LOCs, it is recommended to create small pull-requests which can be managed easily. If there are a lot of changes going in a release, it is better to chunk it down into a number of small pull-requests.

2. Ensure Logical Coherence of Changes

Code reviews are the most effective when the changes are focused and have logical coherence. When doing refactoring, refrain from making behavioral changes. Similarly, behavioral changes should not include refactoring and style violation fixes. Following this convention prevents unintended changes creeping in unnoticed in the code base.

3. Have Automated Tests, and Track Coverage

Automated tests of your preferred flavor — units, integration tests, end-to-end tests, etc. help automatically ensure correctness. Consistently ensuring that changes proposed are covered by some kind of automated frees up time for more qualitative review; allowing for a more insightful and in-depth conversation on deeper issues.

4. Self-Review Changes Before Submitting for Peer Review

A change can implement a new feature or fix an existing issue. It is recommended that the requester submits only those changes that are complete, and tested for correctness manually. Before creating the pull-request, a quick glance on what changes are being proposed helps ensure that no extraneous files are added in the changeset. This saves tons of time for the reviewers.

5. Automate What Can Be Automated

Human review time is expensive, and the best use of a developer’s time is reviewing qualitative aspects of code — logic, design patterns, software architecture, and so on. Linting tools can help automatically take care of style and formatting conventions. Continuous Quality tools can help catch potential bugs, anti-patterns and security issues which can be fixed by the developer before they make a change request. Most of these tools integrate well with code hosting platforms as well.

6. Be Positive, Polite, and Respectful

Finally, be cognizant of the fact that people on both sides of the review are but human. Offer positive feedback, and accept criticism humbly. Instead of beating oneself upon the literal meaning of words, it really pays off to look at reviews as people trying to achieve what’s best for the team, albeit in possibly different ways. Being cognizant of this aspect can save a lot of resentment and unmitigated negativity.

#agile #code quality #code review #static analysis #code analysis #code reviews #static analysis tools #code review tips #continuous quality #static analyzer

Benefits Of C Language Over Other Programming Languages

C may be a middle-level programing language developed by Dennis Ritchie during the first 1970s while performing at AT&T Bell Labs within the USA. the target of its development was within the context of the re-design of the UNIX OS to enable it to be used on multiple computers.

Earlier the language B was now used for improving the UNIX. Being an application-oriented language, B allowed a much faster production of code than in programming language. Still, B suffered from drawbacks because it didn’t understand data-types and didn’t provide the utilization of “structures”.

These drawbacks became the drive for Ritchie for the development of a replacement programing language called C. He kept most of the language B’s syntax and added data-types and lots of other required changes. Eventually, C was developed during 1971-73, containing both high-level functionality and therefore the detailed features required to program an OS. Hence, many of the UNIX components including the UNIX kernel itself were eventually rewritten in C.

Benefits of C language

As a middle-level language, C combines the features of both high-level and low-level languages. It is often used for low-level programmings, like scripting for it also supports functions of high-level C programming languages, like scripting for software applications, etc.
C may be a structured programing language that allows a posh program to be broken into simpler programs called functions. It also allows free movement of knowledge across these functions.

Various features of C including direct access to machine level hardware APIs, the presence of C compilers, deterministic resource use, and dynamic memory allocation make C language an optimum choice for scripting applications and drivers of embedded systems.

C language is case-sensitive which suggests lowercase and uppercase letters are treated differently.
C is very portable and is employed for scripting system applications which form a serious a part of Windows, UNIX, and Linux OS.

C may be a general-purpose programing language and may efficiently work on enterprise applications, games, graphics, and applications requiring calculations, etc.
C language features a rich library that provides a variety of built-in functions. It also offers dynamic memory allocation.

C implements algorithms and data structures swiftly, facilitating faster computations in programs. This has enabled the utilization of C in applications requiring higher degrees of calculations like MATLAB and Mathematica.

Riding on these advantages, C became dominant and spread quickly beyond Bell Labs replacing many well-known languages of that point, like ALGOL, B, PL/I, FORTRAN, etc. C language has become available on a really wide selection of platforms, from embedded microcontrollers to supercomputers.

#c language online training #c language training #c language course #c language online course #c language certification course

Analysis of Merge Requests in GitLab Using PVS-Studio For

Do you like GitLab and don’t like bugs? Do you want to improve the quality of your source code? Then you’ve come to the right place. Today we will tell you how to configure the PVS-Studio C# analyzer for checking merge requests. Enjoy the reading and have a nice unicorn mood.

PVS-Studio is a tool designed to detect errors and potential vulnerabilities in the source code of programs, written in C, C++, C#, and Java. Works in 64-bit systems on Windows, Linux, and macOS. Can analyze the code meant for 32-bit, 64-bit, and embedded ARM platforms.

By the way, we’ve released PVS-Studio 7.08, which was full of new sapid features. For example:

• C# analyzer under Linux and macOS;
• plugin for Rider;
• new mode for checking a list of files.

Mode of Checking a List of Files

Previously, to check certain files, one had to pass .xml to the analyzer with a list of files. But since this is not very convenient, we have added the ability to pass .txt, which makes life much simpler.

To check certain files, specify the –sourceFiles (-f) flag and pass .txt with the list of files. It looks like this:

C#

1

pvs-studio-dotnet -t path/to/solution.sln -f fileList.txt -o project.json

If you are interested in configuring checks of commits or pull requests, you can also do this using this mode. The difference will be in getting a list of files for analysis and will depend on which systems you are using.

Principle of Checking Merge Requests

The main point of checking is to make sure that problems detected by the analyzer do not make it into the master branch when merging. We also don’t want to analyze the entire project every time. Moreover, when merging branches, we have a list of changed files. Therefore, I suggest adding a merge request check.

This is how a merge request looks like before introducing a static analyzer:

In other words, all errors in the changes branch will get to the master branch. Since we wouldn’t like this, we add the analysis, and now the scheme looks as follows:

We analyze changes2 and, if there are no errors, we accept the merge request, otherwise reject it.

By the way, if you are interested in analyzing commits and pull requests for C/C++, you are welcome to read about it here.

GitLab

GitLab is an open-source DevOps lifecycle web tool that provides a code repository management system for Git with its wiki, bug tracking system, CI/CD pipeline, and other features.

Before you start implementing the merge request analysis, you need to register and upload your project. If you do not know how to do this, then I suggest an article by my colleague.

Note. One of the possible ways to configure the environment is described below. The point is to show the steps for configuring the environment needed for analyzing and running the analyzer. In your case, it may be better to separate the stages of environment preparation (adding repositories, installing the analyzer), and analysis. For example, preparing Docker instances with the necessary environment and their usage, or some other method.

To get a better understanding of what is going to happen next, I suggest taking a look at the following scheme:

The analyzer needs .NET Core SDK 3 for proper operation from which the necessary dependencies for the analyzer will be installed. Adding Microsoft repositories for various Linux distributions is described in the relevant document.

To install PVS-Studio via the package manager, you will also need to add PVS-Studio repositories. Adding repositories for various distributions is described in more detail in the relevant section of the documentation.

The analyzer needs a license key to operate. You can get a trial license on the analyzer download page.

Note. Please note that the described operating mode (merge requests analysis) requires an Enterprise license. Therefore, if you would like to try this mode of operation, don’t forget to specify that you need an Enterprise license in the “Message” field.

If a merge request occurs, we only need to analyze the list of changed files, otherwise, we analyze all files. After the analysis, we need to convert the logs to the format we need.

Now, with the algorithm in front of your eyes, you can proceed to writing the script. To do this, we need to change the .gitlab-ci.yml file or, if there is no such file, create one. To create it, click on the name of your project -> Set up CI/CD.

Now we are ready to write the script. Let’s first write the code that will install the analyzer and enter the license:

C#

1

before_script:

2

  - apt-get update && apt-get -y install wget gnupg 

3

4

  - apt-get -y install git

5

  - wget https://packages.microsoft.com/config/debian/10/

6

packages-microsoft-prod.deb -O packages-microsoft-prod.deb

7

  - dpkg -i packages-microsoft-prod.deb

8

  - apt-get update

9

  - apt-get install apt-transport-https

10

  - apt-get update

11

12

  - wget -q -O - https://files.viva64.com/etc/pubkey.txt | apt-key add -

13

  - wget -O /etc/apt/sources.list.d/viva64.list

14

https://files.viva64.com/etc/viva64.list

15

  - apt-get update

16

  - apt-get -y install pvs-studio-dotnet

17

18

  - pvs-studio-analyzer credentials $PVS_NAME $PVS_KEY

19

  - dotnet restore "$CI_PROJECT_DIR"/Test/Test.sln

Since installation and activation must occur before all other scripts, we use a special before_script label. Let me be clear on this fragment.

Preparation for the analyzer installation:

C#

1

  - wget https://packages.microsoft.com/config/debian/10/

2

packages-microsoft-prod.deb -O packages-microsoft-prod.deb

3

  - dpkg -i packages-microsoft-prod.deb

4

  - apt-get update

5

  - apt-get install apt-transport-https

6

  - apt-get update

Adding PVS-Studio repositories and the analyzer:

C#

1

  - wget -q -O - https://files.viva64.com/etc/pubkey.txt | apt-key add -

2

  - wget -O /etc/apt/sources.list.d/viva64.list

3

https://files.viva64.com/etc/viva64.list

4

  - apt-get update

5

  - apt-get -y install pvs-studio-dotnet

#devops #devsecops #csharp #static analysis #gitlab #static analysis tools #static analyzer