Posts

Showing posts from 2020

8 different ways to detect faces in images

Over the years of doing many A.I. algorithms revolving around humans, I needed to detect faces as the first step, Many methods for detecting faces have sprung up by different packages and although they might look to be the same, different applications could make use of the difference between them as some of them are slower than the others while having immunity to occlusion or noise, some of them are slow with no apparent benefit, some are fast but only work on relatively noise free images, some can only detect front faces, some are cpu based and others use the gpu which means not all can be used on all embedded systems like Raspberry pi for example, sometimes you just have the option of using that one package. So it all depends on your choice . Also one seemingly minor factor that can affect detection significantly is the image scale.                                           ...

Use opencv to detect faces using 11 different caffe models

Image
Although many tutorials on using opencv and cv2.dnn.readnetfromcaffe exist over the internet , all of them just use the same model and don't reveal the fact that not all models work the same way. I will be sharing some data related to the performance of  all of those models. You can also check my tutorial on caffe models and how to read them. I, as many others, use opencv mainly for image augmentation or processing. I used opencv for real time applications on embedded systems such as raspberry pi or nvidia jetson. Here i will be showing the results for two aspect ratios 1.0, 1.0/255.0 . The results shown will be the ones with the class human, but i will be showing the overall results as well with a confidence less than the max reported confidence for an object minus 0.3 as i have seen some weird results. Some models work with an aspect ratio and produce completely wrong results on the other. Some models detect a different set of faces on each aspect ratio. I tested...

Flip images in multiple directories for data augmentation with no effort

Image
When preparing a dataset for training a neural network, You would like to fully exploit that dataset to increase your network's accuracy as many papers provided evidence that some of these methods do improve the network's recognition ability, Some of those methods are flipping the images horizontally and scaling the images, all while keeping the original images unscathed. Picture from the oppenheimer's famous words video. Assuming you are using linux or macOS, First you will need to install graphics magick which is super simple sudo apt-get install graphicsmagick we will use the mogrify function from graphics magic to flip and resize images. import subprocess def prepare_dataset():     DIR_PARENTS = ['dir1/',                                     'dir2/',                             ...

Restore corrupted innodb from .frm and .ibd files

Sql comes in many different,  for some flavours, if the db crashes, updated or anything is changed within it. You can simply move the files in the directory of your new sql db and it should read it instantly. With innodb , it does read them but when you try to SELECT from the tables, it returns TABLE does not exist. First install mysqlfrm which we will use to read the tables form. sudo apt-get install mysql-utilities Next do mysqlfrm --user=mysql --server=root:maria@localhost --port=3308 sql_table.frm You can change the variables with the ones specific to your server of course. The output of this will be a command that you can use to create a table of the same form. Create a new db of a name that you desire,  then copy the output of mysqlfrm and execute it to create a table inside our newly created db. Now that a table of the same form is created, Execute those commands ALTER TABLE 'sql_table' ROW_FORMAT=DYNAMIC; ALTER TABLE sql_table DISCARD TABLES...

Create a route optimization algorithm with zero costs using google's OR-tools and OSRM Part 4

Image
We continue with our series  and we create a function that takes care of reading the output of the algorithm, converting it to a readable form and plotting the routes on the map. def print_solution(data, manager, routing, assignment):       Route = []       for vehicle_id in range(data['num_vehicles']:                 index           = routing.Start(vehicle_id)                 idx               = manager.IndexToNode(index)                 plan_output = 'Route for vehicle {} with ID {} and capacity {}:\n'.format(vehicle_id, data['IDs'][idx], data['capacity'][vehicle_id])                 Route[Iter].append([data['locations'][idx], data['IDs'][idx]])                 prev_ind...

Create a route optimization algorithm with zero costs using google's OR-tools and OSRM Part 3

This is a continuation to the route optimization series . First we define the function that will be used by the optimizer to get the distances between points.   def distance_callback(from_index, to_index):                 from_node = manager.IndexToNode(from_index)                  to_node      = manager.IndexToNode(to_index)                          if from_node == to_node:                                return 0                          return data['distance_matrix'][from_node][to_node] IndexToNode is used to correlate between a location as a node index inside the being tested/optimized route by the solver and the index of the loc...

Create a route optimization algorithm with zero costs using google's OR-tools and OSRM Part 2

We have covered querying the server for data about events, offers and requests in the first tutorial . Now we start building the data to be fed into OR-tools. Offer_df      = False Request_df  = False while type(Offer_df)     == bool or           type(Request_df) == bool:                 df = read_events()                 Offer_df = read_offers(df['event_id'])                 if type(Offer_df) != bool:                        Request_df = read_requests(Offer_df['event_id']) The reason why we check for the type of Offer_df or Request_df  instead of checking if they are equal to False is that if Offer_df is a numpy array, then python will return an error as comparing a numpy array to a value is invalid and it will ask us to be more ...

Create a route optimization algorithm with zero costs using google's OR-tools and OSRM Part 1

Image
This is a tutorial for an algorithm that has riped over the years of experience in doing algorithms and websites  for carpooling, Uber like service, Freight routing, and many other applications. I will be stating the current algorithm and i will discuss my decisions on certain code implementations as well as some of the previous decisions that i took and proved to be wrong, slow or error prone. This algorithm is capable of optimizing for thousands of locations, and made to be highly customizable. Features of the algorithm are: Reading data from and exporting data to an excel sheet or a Database. Multiple time windows per location. Multiple vehicle types, capacities, buffers, depots. Custom optimization parameters like distance, time, fuel consumption, etc. Pickup and dropoff. Custom loading and unloading time per stop. Custom service time at certain locations. Custom max idle time waiting for the next stop's time window. Custom distributed workload amongst ...