I was given the chance recently to use [netsparker](netsparker.com) cloud and wanted to make it further work for me. I'm proud of the dev-ops pipeline where I work now and figured, "why not include netsparker?". This is a great idea! However, no tool was built at the time to successfully do this. Luckily as a software engineer, I can do something about it.

That's why I build the netsparkerscanrunner. I wanted to trigger a scan against my public facing API (development, QA, staging, and production) and see if I'm making the environment better, worse, or keeping it secure. I also know that other people in my company might want to use this tool and we don't all use the same build system. That's why I chose to build it with node and use the JUnit test output. This seemed like the most universal. To run it in your build process all you need to do is

npm install -g netsparker-scan-runner
netsparker-scan-runner -u MY_NETSPARKER_ID -t MY_API_TOKEN -p MY_TEST_PROFILE -s https://MY_ENVIRONMENT_URL -j results.xml

Then just upload results.xml as a JUnit file and you're done! It's that easy. Crazy right? No way. Here's how it works.

Here is an example of the output you might see in the console

2019-04-05T01:37:26.3608115Z Starting scan...
2019-04-05T01:37:26.8879679Z Scan triggered!
2019-04-05T01:37:27.0604678Z Scan running - 0/0 complete
2019-04-05T01:37:32.2251357Z Scan running - 0/0 complete
2019-04-05T01:37:37.3878272Z Scan running - 0/0 complete
2019-04-05T01:37:42.5585536Z Scan running - 0/0 complete
2019-04-05T01:37:47.7330133Z Scan running - 0/0 complete
2019-04-05T01:37:52.8977720Z Scan running - 0/0 complete
2019-04-05T01:37:58.0716098Z Scan running - 0/0 complete
2019-04-05T01:38:03.2414409Z Scan running - 0/0 complete
2019-04-05T01:38:08.4273151Z Scan estimated start time - 5
2019-04-05T01:38:13.5985996Z Scan estimated start time - 5
2019-04-05T01:38:18.7714179Z Scan running - 0/0 complete
2019-04-05T01:38:23.9343766Z Scan running - 0/0 complete
2019-04-05T01:38:29.0988487Z Scan running - 0/0 complete
2019-04-05T01:38:34.2556198Z Scan running - 0/0 complete
2019-04-05T01:38:39.4235132Z Scan running - 140/142 complete
2019-04-05T01:38:44.5983008Z Scan running - 140/142 complete
2019-04-05T01:38:49.7683913Z Scan running - 381/381 complete
2019-04-05T01:38:54.9360558Z Scan running - 381/381 complete
2019-04-05T01:39:00.1130823Z Scan running - 675/675 complete
2019-04-05T01:39:05.2727981Z Scan running - 675/675 complete
2019-04-05T01:39:10.4357841Z Scan running - 1381/1381 complete
2019-04-05T01:39:15.6018347Z Scan running - 1381/1381 complete
2019-04-05T01:39:20.7795993Z Scan complete!  Completed Steps - 1381
2019-04-05T01:39:20.9737958Z IssueUrl                                                                         Title                                                          Type                        Url                                      
2019-04-05T01:39:20.9738382Z ------------------------------------------------------------------------------------------------------------------------------------
2019-04-05T01:39:20.9738615Z ISSUE_URL                                           Forbidden Resource                                                        ForbiddenResource          URL
2019-04-05T01:39:20.9738801Z ISSUE_URL                                           Missing X-XSS-Protection Header                                MissingXssProtectionHeader  URL
2019-04-05T01:39:20.9738963Z ISSUE_UR                                            Expect-CT Not Enabled                                                  ExpectCtIsMissing           URL
2019-04-05T01:39:20.9739227Z ISSUE_URL                                           HTTP Strict Transport Security (HSTS) Policy Not Enabled       HstsNotEnabled             URL
2019-04-05T01:39:20.9742029Z ISSUE_URL                                           Insecure Transportation Security Protocol Supported (TLS 1.0)  TlsVersion1Support          URL
2019-04-05T01:39:20.9742190Z 
2019-04-05T01:39:20.9742445Z Generating jUnit report...
2019-04-05T01:39:20.9742582Z jUnit report generated
2019-04-05T01:39:20.9742720Z Netsparker Scan Runner Complete!

Netsparker Cloud has a wonderful API that you can use. This npm just does all the heavy lifting for you around this API and JUnit.

If a test doesn't pass with netsparker, those are presented in JUnit. Here is a example of these issues showing in Azure Devops.

Again, easy right?

Security should be apart of any application you code or create. In today's world, you simple can't ignore it. If you have any issues with this tool, please create an issue. I'd love to hear your feedback and suggestions!

If you want to hear myself and Ricky talk about security in applications, be sure to check-out episode #32

DevOps should be at the core of every development team. Teams with a high performance DevOps cycle can deploy more frequently, get products to the market faster, increase revenue, and have lower failure rates. The reason most teams don't invest in DevOps is because it can be intimidating. Microsoft has made that easy with Azure DevOps by taking the hassel out of mainting builds, releases, code, and user stories so you can focus on the product.

What is DevOps?

"DevOps is the union of people, process and products to enable the continuous delivery of value to your end customers"

The diagram below shows a great example of what a good DevOps system can do.

What Can I Achieve with DevOps?

High performance DevOps companies can acheieve...

• 46x Deployment Frequency
• Faster Time to Market
• 7x Lower Change Failure Rate
• 2,555x Faster Lead Time for Changes
• Increased Revenue
• 2,604x Faster Mean Time to Recover

Source: 2018 Accelerate: State of DevOps: Strategies for a New Economy." N. Forsgren, J. Humble, G. Kim. DevOps Research and Assessment (DORA)

What technologies do I need to support DevOps?

Now that we know what DevOps is and what I can achieve from it, how do I go about supporting it? This is normally when people start to the feel the burden cause now they need to manage more VMs, more applications, just to help them manage their own applicaiton.

Continuous Integration (CI)

CI helps teams move faster and improve overall quality. Each time you commit code or submit a PR you can make sure it doesn't break your builds and still passes your tests.

Continuous Deployment (CD)

CD bridges the cap between a product release and CI. It allows you to automate the whole process from committing code to release if all your CI/CD tests pass successfully.

Continuous Learning & Monitoring

Using tools like Application Insights you can identity how your application is performing and see if recent reployments made things better or worse.

Show me Azure DevOps

Azure Boards

Boards allow you to plan and track items across your team using agile or scrum approaches. You can use tools such as Kanban boards, backlogs, team dashboards and custom reporting. This tool allows you to connect your ideas at every development stage and keep your team aligned with code changes.

Azure Pipelines

Pipelines allow you to build, test, and deploy with any language, platform, and/or cloud provider that suites you. You can pick from Windows, Linux, and even macOS build agents when it come time to run your code. Whether your project is in C#, Java, Ruby, Javascript, PHP, Android or iOS you will be able to build, test and deploy using pipelines. In your build, you can also use tasks built by the community! If it doesn't come included, you can find it or even build yourself. And best or all, the code doesn't need to be in Azure DevOp! You can use pipelines for your open-source project on GitHub with nearly no limits.

Azure Repos

Similar to GitHub repos, but you can get unlimited private repos with all the same tools and advanced file management. Not only is is a codebase your used to, you can still continue to use your favorite IDE, editor and git client along with having access to Web Hooks and API integrations.

Azure Test Plans

Before shipping code make sure the correct test plans, whether manually or automated, run correctly. Having a good tool to manage all your tests lets you capture rich data and get end-to-end traceability. You want to make sure your team finds the issues before your users.

Azure Artifacts

Share packages with your team and even make them available for CI/CD pipelines with a single click. Whether you are used to having packages come from maven, npm or nuget, you can manage and create them all right in your team allowing for bettering sharing of code and resources.

Examples

Setting up an existing GitHub repo with Azure Pipelines can be done in minutes. If you want to see an example check out this project.

Learn More

If you are ready to get started or even want to learn move, head of over to https://azure.com/devops. You can get started for free and grow with their reasonable pricing structure.

For entire slide deck, you can download it here and of course of if you have any questions or need help getting started please feel free to email me at nathan@theengineerofwhat.com.

Recently, I was able to give a demo at my local user group showing how to get start with Mobile UI Automation testing using the Appium framework. I figured I would share this information with out listeners and whoever else might come across this. At the end of this, the slide deck along with the source code will be available.

4 Types of Testing

For a mobile app to be tested correctly, we need to make sure we cover all areas. Mobile apps normally have some sort of communication channel with an API or authentication service. We want to make sure all of this works.

Unit Testing

Unit tests are easy to write as developers. They are designed to test smaller pieces of your code and are extremely helpful for quick regression.

Performance Testing

Performance testing is normally the type of testing that gets ignored, but it can be very important once your app starts getting thousands of users. Not only do you have to check the API, but you have to make sure the your app will run on lower performing hardware.

API Testing

Testing you API can also be done with Unit Tests, but again is something that can get overlooked. It helps you check endpoints during a regression without having to hook your app up to it.

UI Testing

This is the type of testing we will be covering with Appium. UI Testing allows us to test actions and features as a user would be using them.

Get Started

Appium is what we will be going over in the rest of this article. It is an open-source automation framework meant for testing iOS and Android native, hybrid and web apps in essentially any language.

App Center by Microsoft allows you to continuously build, test, release, and monitor apps for every platform. They support building your app whether written natively (swift, objective-c, or java) or with a cross platform solution (xamarin or react). You can also run UI tests across multiple devices at the same time using appium.

AWS Device Farm is a testing service that lets you test your Android, iOS and web apps across multiple devices. Just upload your app and test suite and away they go.

What is Appium?

So this is great so far. I know what I need to test, the approaches I can take to test my app, but what is this Appium thing and how do I use it?

Appium is an open-source tool for automating native, mobile web, and hybrid applications. It is based off the Selenium WebDriver which allows these tests to be written in almost any language and be cross platform. That's right, you read that correctly. Not only can I write the same line of code to run a test on an iOS device and an Android device, I can write that desk in almost any language! Some of the supported languages appium has listed on their website and we have used during this demo...

• Javascript
• Java
• C#
• Python
• Ruby
• PHP

Crazy right? How does Appium accomplish this? Appium works by starting an HTTP server written in Node.JS. This server handles sessions to iOS and Android devices for you. So for example, in my Appium tests if I want to tap a button I would write the following.


MobileElement el1 = driver.GetElementById("MyButton");
el1.Click();


This generates a JSON message and sends it over HTTP to the Appium Server. Appium then knows based off the session it created when I made my driver that this is an iOS or Android device. If an iOS device, it will communicate the to the Apple Instruments Command Client and run tests on the device using XCUITest (iOS 9.3+) or UiAutomator (< iOS 9.3). If this was an Android device, it would communicate with Googles UiAutomator2 command server and run the needed tests.

Appium in Action

Let's see real quick how this would work. I wrote a simple test for a Weather App written in Xamarin we used for Xamarin Dev Days. All it does it open the app, click a button that says "Get Weather", then grab a screenshot. Let's see how it works.

Woah! That was quick and easy right? Let's look at the code.

[Test]
public void GetWeatherTest()
{
elements.GetWeatherButton.Click();
Thread.Sleep(5000);
var screenshot = driver.GetScreenshot().AsByteArray;
var filePath = Path.Combine(TestContext.CurrentContext.TestDirectory, "Weather.jpg");
using (var fs = File.Create(filePath))
fs.Write(screenshot, 0, screenshot.Length);
TestContext.AddTestAttachment(filePath, "Weather Results");
}


Easy right? See any specific code for iOS or Android? That's cause there isn't (well, not really). The only code that is specific that you can't see is connecting to the device. This is only need to be done once, but once it is all tests are the exact same!

Writing Tests

As a developer, what I'm doing is pretty simple. Writing code, running in on devices, I do this every day. What about people who don't write code? How can I expect them to write tests?

Appium has an answer for this. It's called Appium Desktop. This application that runs on both Windows and macOS will allow anything to connect to a device, open/install an app and then record actions to write a test! Yes it's actually that easy. Don't believe me? Let's write a test that changes the text bar from 'Seattle, WA' to 'Albany, NY' and then get the weather.

As you can see, Appium Desktop lets you start the Appium server, connect to a device, and then inspect and control the elements on the page. You can record these events which will then translate to Javascript, Java, Python or Ruby code automatically.

I didn't cover how to setup your device with Appium, but this is actually really easy. Before you start a session, you need to define device capabilities based on how you want to connect. Appium does a great job listing the ones you need and maybe the ones you don't but would be helpful to know here.

Taking these tests to the cloud

I have my app. I used Appium Desktop to write my tests. I tested them on a device I have on my desk. Now how do I test a bunch of devices at one?

Using AWS Device Farm you can tests tens, even hundreds of devices at the same time to make sure you app performs as it should no matter what the hardware is. Doing this is super easy. If you want to see how to write the tests, you can follow AWS tutorial here and/or use the tests I wrote in the Github link below.

After you create and upload your app and tests to the AWS device farm, you can select how you want to run your tests. The first 1,000 minutes are free, so feel free to give it a shot first. You can select the exact type of devices you want and even the configurations of the device such as if WiFi is enable, throttling, etc. I choose 5 devices for this example.

• Amazon Fire HD 7
• Google Pixel 2
• LG Nexus 5
• Motorola Moto X
• Samsung Galaxy S9

AWS gives you a lot of information to verify your tests. It runs across all devices at the same time, prints passed/failed tests, video playback, devices logs, device performance and more. You can see how your app performance at a quick glance or drill down into a specific device to see more information.

As you probably have noticed, a few tests have failed. Why would a test fail for one device over another? For my "get weather" button, I'm using XPath locators. XPath locators are '''not''' reliable and as shown can have issues across devices and versions of androids. Best practice for Android and iOS is to use an identifier. This will ensure finding that element on the 'page' without these issues across devices.

Wrapping up

UI testing is normally something that gets over looked, but with mobile development it becomes almost needed. iOS has a handful of devices, but Android has over 19,000. Even though most share the same screen size and other elements you want to make sure you cover as many as possible for your users. UI testing can save time on a smaller team and allow you to scale testing as your app continues to grow. Using Appium allows your tests to work for your team not against. Since Appium allows the same code to work on both Android and iOS devices as well as being written in any language, you aren't locked in to a specific set of rules if your development changes. It allows you to move just as fast as your product.

If you want to see the slide deck from the user group meeting, you can download it here. To see all the code available and try it out for yourself fork it from github. Of course of if you have any questions or need help getting started please feel free to email me at nathan@theengineerofwhat.com.