big data
big data
 

Today, data collection is an important part of any business. Good data can help proactively address issues, measure progress and capitalize on opportunities such as promoting innovation and higher productivity. However, solely collecting data is insufficient to drive efficiency, change, and innovation, and this is why we also need to analyze that data.

Throughout this article, we aim to demonstrate how good data interpretation can help save time and increase productivity.

Our aim will not be to dig too deep into technical details; the main objective will be to understand how a good design and solution can bring value to customers.

In doing so, we will refer to one of our successful projects, for which we have used Matlab to develop the solution. However, it is worth mentioning that any other more popular programming language (Python, Javascript) could have been used.

The project began when our client, a well-known company in heavy industry, specialized in agriculture equipment, faced a significant challenge. Due to the silicon crisis, they changed the initial design in terms of equipped sensors, selecting new ones, which were not 100% compatible with the existing software and toolchain.

The team started with the data collected from the new sensors and an existing tool that was displaying plots/ graphs based on old sensors’ data. The new data was proprietary, and a library provided by the manufacturer of the sensors was used to convert it to a readable format by the Matlab programming language.

Because there were some major differences between these two types of files, a second tool to convert and manipulate data was necessary to keep compatibility.

The focus was on making the new data compatible with the existing software and making it easy to change, trim, and analyze before exporting it to the old format.

Thus, the team decided to create two separate tools: one for converting and manipulating data and one to improve on the existing script (which was generating the plots for the old data).

Afterwards, a three-step plan was implemented:

  • identify the best solution for the GUI interface for both tools.
  • create the solution to manage the differences between the new files and the old ones.
  • find a way to manipulate data.

the GUI interfaces.

Luckily, Matlab offers an intuitive and easy-to-use application for that.

tabel
tabel
 

This offers basic features like:

  • Human readable data viewing
  • Renaming sensors data to predefined names so that they match old software
  • Graphic visualization
  • Trimming files capabilities
  • Calculating baseline values from a set of values
  • Display for multiple sensors on the graph
  • Zoom in/out selecting the start and stop points on the graph for trimming.

managing the differences between two types of files.

The old files were structured as a table with two columns: one for timestamps and one for the values of the sensor.

The new ones had only one column with the data and a description that contained the frequency of the measurements, starting time, and end time.

Based on the frequency and starting time we generated the first column for the timestamps and made a table matching the format of the old format type.

manipulating the data.

Manipulating data was the most important part of the tool.

There are two different tabs for it.

First, we needed to rename columns (channels) generated by the conversion to predefined names, making it more intuitive for users and exporting the changes to a new file.

 

unnamed1
unnamed1

Second, we needed to trim and visualize the data. On this tab we could add/trim/mark/correlate points in time with locations on the map, correlate locations on the map with points on the graph and trim data based on a part of the track or part of the graph.

grafic
grafic
 

As the new data format had GPS data in it, adding a map for better visualization, usability and more precise analysis, and adding features like trimming the data based on parts of the track, which could be selected from a map or trimming data based on graphs became an important part of the tool.

 

map
map
 *not the original map  

Map view features:

  • Selecting a start and stop point which will update the map and the graph in the tool with the corresponding part of the track
  • Selecting a part of the graph will update the map accordingly
  • Zoom in/ Zoom out / Automatic zoom
  • Restore to the initial view

Implementing the map part was quite interesting. Initially, it’s implementation was done in Javascript, being the most accessible to be used with google maps APIs.

Nevertheless, in our research we came across some great work, done by Zohar Bar-Yehuda, https://github.com/zoharby/plot_google_map.

He implemented a Matlab library, which is displaying google maps as a background on a plot with longitude and latitude as x and y and all the other features are tools available in Matlab.

With the complete first part of the tool, now the manipulated data could be exported and used on the existing software to generate reports.

This tool made it possible to reduce work labor costs from 40 hours for a skilled developer to 16-24 h of a regular operator, as our customer reported.

With the generated map and including a report generated by the existing tool, now data could be easily analyzed to identify obstacles and optimal combine parameters (pressure, crane angles, speed, etc.) on tracks and fields helping our customer to better understand and decide on an optimal course of action.

Overall, our project was successful in making the new data compatible with the existing software and providing a user-friendly solution for changing, trimming, and analyzing the data before exporting it to the old format. By demonstrating how data analysis can increase productivity in the agriculture industry, we hope to inspire others to adopt similar approaches in their own businesses. Good data interpretation can save time and resources, and by implementing effective solutions, businesses can optimize their operations and stay ahead of the competition.

about the author
Emanuel
Emanuel

Emanuel Mazilu

system debug engineer

Emanuel joined Randstad Digital two years ago as a software developer. Since joining the company, he has been working on an automotive project and developed a software solutions on another. He is always seeking out innovative solutions and is enthusiastic about sharing his knowledge and experience with others. These qualities have helped him to excel in his role at Randstad Digital, where he is dedicated to delivering top-quality software solutions to his clients.