walk-stats (part 2)

Let’s configure some services

Changing .gitignore

I’ve developed different projects with Java, git and Eclipse  and I’ve come up with a .gitignore I use in my projects, it’s quite useful and it’s becoming more complete with time, so It’s something I just put it there and everything it’s fine committing.

Here it is:

 Keeping things logical

When I work on a project I tend to get stuck on little details instead of focusing on keeping the work flowing and worrying about the details later.

A nice way I found to solve this it’s using tools designed to help keeping track of what needs to be done and how.

Issues and milestones

I like github‘s issues and milestones mechanism to plan what to do and when to do it, you can find more about this in the linked documentation.

It’s actually much more than this, an issue tracker for people to discuss and plan integrations, bug fixes and ask questions about a project.

Yes it’s pretty cool stuff and for open source public projects it’s free to use under the github terms of service.

Planning

As I go along I’ll plan to release this series of post about developing walk-stats directly here.

Here is this episode milestone.

I’ve created different issues for the arguments I want to talk about on this episode, it reflects the needs of the project too, so here we go:

Choose and configure a build automation tool for the project

There are various ways to write and compile a program in Java as in every other language.

You can simply create a class then call javac on the source code and java on the class to run it.

This is all fine and dandy, teaches the basics of how interpreted and compiled languages works and the steps needed to create something executable from just text.

Anyway, as the project you’re working on becomes more complex than a simple main method it’s much easier to manage it with a build automation tool.

This tools are useful because they minimize the chances of forgetting something (a machine does what it’s supposed to do instead of a human), and most of them also open up an opportunity to stand on the shoulders of giants, giving you automatic access to a huge library of already made open source projects to use as dependencies to solve problems people have already analysed, tested a kindly shared with the world.

Examples of build automation tools are:

Each one has its nice things and its weakness, the perfect one does not exist.

I’m pretty much familiar with Maven, I’m using it at work and makes a lot of sense, but I’m willing to try out Gradle, it’s a young tool, powered by a Domain Specific Language powered by the language Groovy.

Everybody it’s talking of it as the solution to all evil, so a look at it sure can’t hurt.

Cloning the repository on my local machine

The first thing to do is cloning the repository from github to my local machine.

I won’t focus on the procedure to install and configure or usage of git on any platform, it’s well detailed in the git documentation.

Let’s start actually adding something to this empty and lonely project🙂

git clone https://github.com/scompo/walk-stats.git

Let’s change directory into the project itself with

cd walk-stats

An easy way to start using gradle it using its wrapper so let’s configure the project to use it (again, I suppose you have a working installation of gradle on your machine, more information on how to install it can be found here).

gradle wrapper

will generate a directory structure like this:

│ .gitignore
│ gradlew
│ gradlew.bat
│ LICENSE
│ README.md

├───.gradle
│ └───2.11
│ └───taskArtifacts
│ cache.properties
│ cache.properties.lock
│ fileHashes.bin
│ fileSnapshots.bin
│ outputFileStates.bin
│ taskArtifacts.bin

└───gradle
└───wrapper
gradle-wrapper.jar
gradle-wrapper.properties

If that’s the case we’re good, it will be also a good time to commit the changes on a branch as I did here.

Let’s then init the project with a build.gradle file and commit again with

gradle init

And you finally should have all the files I have here.

Creating a stub of the project for further configuration

Let’s start writing some code, nothing fancy, pretty boring I’d say.

I just want to create a Java class and a couple of test cases that fails and pass  to have a basic structure for the project and get it compiled for the next steps.

Packages

The first way to keep the code readable it’s organizing it in packages, this helps understand where to find what you need and it helps also not having a horrible mess of files around in the main folder.

Lets create some folders:

mkdir src/main/java/com/github/scompo/walkstats
mkdir src/test/java/com/github/scompo/walkstats

Now let’s create an empty Java Class and a JUnit test for it.

Here’s Application.java

And here’s ApplicationTest.java

I know, it’s pretty sad for the first time I wrote some code on this project, but now I’m ready to move on with the remaining phases for the configuration of the project.

The project now should look something like this.

Setting up a Continuous Integration testing service

Setting up a Continuous Integration testing service used to be a pretty demanding operation to do.

Luckily for everybody today services like Travis CI exists, they make a developer’s life much easier! (especially on open source code hosted on github😉 )

To prepare the project to be under CI testing at each commit just sign in with your github credentials in Travis CI, click on the plus icon and flick the switch on the project.

The next thing to do is creating a .travis.yml and place it in the project root folder.

There’s actually not much about it apart from telling travis that this is a java project and that I wanted to use the oracle JDK version 8 to compile it.

Mine looks something like this:

Well, commit and push, and what do you know, the first build failed like this.

No problem, stuff goes wrong all the times, I’m used to it, it’s one of the ways you can learn something.

The interesting part is the following:

$ ./gradlew assemble
/home/travis/build.sh: line 179: ./gradlew: Permission denied

The command “eval ./gradlew assemble” failed. Retrying, 2 of 3.
/home/travis/build.sh: line 179: ./gradlew: Permission denied

The command “eval ./gradlew assemble” failed. Retrying, 3 of 3.
/home/travis/build.sh: line 179: ./gradlew: Permission denied

The command “eval ./gradlew assemble” failed 3 times.

At least I can’t say that he didn’t try it😛

Jokes apart it looks like the ./gradlew file hasn’t got the right permission to be run by Travis CI.

You can find more details about it here.

The solution is to add to .travis.yml some pre build commands to set it executable, to make it look something like this:

Now the build still fails, but just because of the failing test I created!

This is what I call a success🙂

Setting up a test coverage service

The next thing to do is setting up a test coverage service, it’s nice to know that your code is covered by tests, this gives you confidence in what you’re doing and prevents unwanted collateral regression.

First of all let’s remove the useless failing test from ApplicationTest because I don’t want my tests to fail for no reason anymore leaving just passingTest.

We are now good to set up a coveralls.io test coverage listener for this project.

The procedure to do this is similar to the one illustrated for Travis CI.

Log in with the github credentials, click on add repos and add the one for the project.

To keep things simple I choose to use the coveralls-gradle-plugin.

I needed to do some editing to a couple of files:

.travis.yml again to add a after_success phase to post the results to coveralls.

build.gradle to add the jacocoTestReport task

It worked, of course there’s no code so the report it’s pretty sad, but this means nonetheless that everything’s talking together as expected.

Setting up a code analysis service

This is probably a bit anal…

Anyway, it’s nice to have an eye on how your code is written, typos could result into vulnerabilities or headaches later.

I’m going to use Codacy to analyze my code.

The procedure to set this service up it’s basically the same as the 2 I discussed earlier on.

Log in with github credentials, choose the repository and enable it.

Here’s the result of all my efforts.

Summing things up

I’ve configured all the stuff I wanted as a service on my project.

A nice touch is to add badges to the readme.md file to see the status live from github

This is the result.

After all this configuration stuff I think it’s time to write some code, tune in next time for data modeling and other fancy things.

You can find the code, like it has been written at the time of this post here.

See you later.

Permission error with Gradle on Travis CI

What happened?

I was setting up the continuous integration for a project that used Gradle as build tool on Travis CI.

The first build failed due to a repeated permission denied error running the gradlew wrapper file.

The important part it’s the following:

$ ./gradlew assemble
/home/travis/build.sh: line 179: ./gradlew: Permission denied

The command “eval ./gradlew assemble” failed. Retrying, 2 of 3.

/home/travis/build.sh: line 179: ./gradlew: Permission denied

The command “eval ./gradlew assemble” failed. Retrying, 3 of 3.

/home/travis/build.sh: line 179: ./gradlew: Permission denied

The command “eval ./gradlew assemble” failed 3 times.

The solution

Obviously the first thing to do is searching on google about the problem.

This lead me to this stackoverflow answer, from wich I used the second solution proposed.

Basically the problem is due to the fact that the the gradlew wrapper file it’s not set as executable.

I liked more the solution of adding a before_install call to chmod +x on the file in question.

So from this .travis.yml

I just ended up with:

Which is fine.

Bye.

walk-stats (part 1)

Let’s develop something

Introduction

Sometimes you need to keep yourself busy doing something, so I decided to document the birth and evolution of my next project from the early beginnings to a working stage (hopefully).

I will develop an application to keep track and create statistics on walks I make around here when I have some free time.

There are already programs that basically do the same thing like map my walk, walk log and so on.

I know it’s not nice to answer a question with another one, but to the question why another one? I would simply answer, why not?

This could be an interesting project and I haven’t seen much documentation about the process of developing a project from the beginning.

I’ll try to be as clear as I can and document each choice I make to make some sense of this in the end.

I think doing this could:

  • Keep me busy, making me think about something else.
  • Be an interesting challenge to solve.
  • Make me learn something new.
  • Let me experiment on things.
  • Make my coding skills better.
  • Help somebody who’s curious about anything I’ll talk about here.

First things first!

Well I like to do code while listening to music, so the first thing to do is finding something to listen to.

Getting back home from work I was thinking about the record I.V. by the band Loma Prieta.

I don’t have it on my phone so the choice it’s pretty easy on what to listen to for me, you can buy it by clicking on the links.

Here’s the player from bandcamp if you want to check them out:

What do I want to do?

Programming basically consists of finding ways to solve problems and make a computer do it for you in the best way possible.

What I want to do is to keep track of when I walk and calculate statistics to see how things are going.

That’s basically it for now, later on I’ll try to analyze this in more details and try to find a solution each subproblem to come up with something that would work for me.

Decisions, decisions, decisions…

How to write it?

I like the Java programming language, I also use it every day at my job, so the choice for me it’s pretty easy, using it🙂

How to call it?

Well this should keep track of my walks and do statistics on them, why not calling it walk-stats?

Sounds reasonable, and it’s not already taken, so be it, welcome walk-stats.

How to release it?

I’m a pretty strange person, I’m a bit of a misanthropist, but I still believe that helping someone who needs it (and while programming you came across a lot of challenges) it’s a nice thing to do.

I also think that more people looking and collaborating at something would make it better.

So I’m releasing the code for this project with the BSD 2-clause license, basically, do whatever you want with this program as long as you keep the license for the code I wrote in source and binary form.

Analysing the problem a bit more in depth

A nice way to attack and solve a problem it’s dividing it in smaller subproblems, as a lot of people said before: divide et impera, it worked then, and still works today.

So from keep track of when I walk and calculate statistics on it, it’s easy to divide it into 2 smaller problems:

  1. Keep track of when I walk.
  2. Calculate statistics on it.

This is helpful  because I can focus on the first problem first and move on with the second part later.

Keep track of when I walk

Questions? Answers!

How can I record data from a walk?

Well, I don’t feel like writing a full pledged gps logger application for my Android phone, I just want to aggregate the data in a unique place and make some computations about it, so Logger GPS for Android would sure do.

It’s open source and the code it’s available on github, free to download from the Google Play Store.

With it I can save data in the GPX format, which is pretty cool.

What’s the data I need to save about each walk?

For now, without worrying too much about it I guess I need to save:

  • The date and time I was walking
  • The actual data GPS positions from the walk

Where can I store that data to make it easily accessible and usable?

Probably in a database, where I can structure the data and query it.

How do I want to see the data?

Well it would be cool to see the actual map with the track on it, so I’ll try to keep it an option. But it’s good enough to see the numbers too as the first release.

Calculate statistics on the data

Other questions and answers

What do I need to compute from the saved data?

It should be useful to see:

  • Total time.
  • Total distance.
  • Average speed.
  • Maximum instantaneous speed.

All of this also from all walks together. (this should be be easy from the single aggregated data).

How do I want to see the data?

Again, graphically a graph would be cool, I’m not a graphic so I would keep that in mind, but just the raw data it’s good enough for the first release.

Let’s start setting up the project

Distributed version control system for the code

I think it’s important to keep the code under version control, it helps focusing on what you’re doing, and all the others nice things like distributed development, versioning, history, rollbacks and so on.

I like git, my code is open so I will use github as the public repository for this project.

I won’t focus much on how git or github works, if you need help there’s a nice documentation about them.

It works like this:

  1. Create a github account
  2. Login
  3. Create a new repository
  4. Add a .gitignore and/or a license and/or a readme if needed
  5. Profit

So here it is, the repository for the project!

That’s it for now, but the fun part is yet to come🙂 See ya!

JUnit test class names are important with maven

maven-logo-black-on-white

junit-logo

I came across a funny issue on utils (available here) today, basically 2 unit tests were not executed by Apache Maven on Travis CI.

It’s funny beacause it happened to me before, so I knew how to fix this problem.

On my machine I usually run JUnit tests with the JUnit plugin integrated into the Eclipse IDE because it’s nice, green/red, failure log, stacktrace and stuff.

Instead, when calling from the command line mvn test, Maven uses the maven-surefire-plugin to execute the tests. In its documentation it says that the default configuration, which I can’t be bothered to change, is to pick up every file name that matches this expressions:

  • **/Test*.java
  • **/*Test.java
  • **/*TestCase.java

A typo like NullCheckerTests, instead of NullCheckerTest prevents the tests in the class from being executed because the class it’s just ignored during test execution.

The solution it’s easy, following naming conventions, I think it’s nice to call all my test with an ending *Test in the name of the file and class, after all if you’re declaring an xxx test class, so it should be called XxxTest in a file named XxxTest.java.

An example of the problem and the relative solution can be found here.

I thought that sharing this issue could be useful if anybody encountered the same problem.

By the way, because of this issue I released version 1.0.5 of utils.

Utils

Long time, no see.. It’s been a very busy year to say the least.

I always liked to solve problems, to write code and experiment with stuff, so as you do, I developed some personal projects to try out new things.

I usually develop in Java, using Apache Maven as my build and dependency management tool, in my spare time and at my job too. I’ve noticed that I usually keep my general purpose utilities classes under packages like *.utils or *.helper.

This led to a lot of copy-pasted code all over the place. A lot of methods were also untested. I couldn’t even be bothered to look inside the other projects if there was already what I needed, so I ended up re-writing already existent stuff, that’s for sure.

I finally decided that I’m done with this mess.

I started a project called utils, and I choose that it should be open source and available under the BSD 3-Clause License.

The source code is hosted on GitHub and I put the project under continuous integration testing with a service called Travis CI. I use another service called Codacy that provides automated code analysis. I also discovered another service called JitPack looking around for a quick way to share my artifacts online. The home page of JitPack reads “Easy to use package repository for Git” and I can say that’s really great! It uses maven as the build tool, provides artifacts for sources/javadoc and it’s integrated with GitHub release system (just tag your code on git and you’re good to go basically, artifacts are online usually the order of seconds).

All this services are free for open source and it’s a great technological stack to write good code, check them out.

Here’s what i’ve setup for the project:

utils is open source and in development so, of course suggestions, critics, contributions and whatever are welcome.

Soundcloud API test.

I’m planning to develop a web application to search for free music and stream it online from various sites.

The first one I thought about was soundcloud that gives you access to their APIs here.

So I just pulled up this little python script to test it out.

It uses the soundcloud-python package wich is downloadable with pip or easy_install.

pip install soundcloud

easy_install soundcloud

just edit the client_id='YOUR_CLIENT_ID' line with the actual client ID. And try it out with a random track, pretty easy. Check the terms of use, of course.

More to go🙂