DevOp - consultant - artisan

Qualité, amélioration continue, agilité, professionnalisme.

Un besoin ? Un projet ? Écrivez moi !

debian, ubuntu under KVM really slow

Share Button

I can’t remember where I found this, so, if you are the original author of this solution, say hello, with the original link 🙂

If you install a debian or an ubuntu, you can experience some bad perf.

I use virt-manager to manage my vm’s.

Simple file with some options for the QXL driver fix the problem.

nano /usr/share/X11/xorg.conf.d/qxl.conf

Inside the file :

Section "Device"
    Identifier "qxl"
    Driver "qxl"
    Option "ENABLE_SURFACES" "False"
EndSection

And reboot !

 

Share Button

How to : elastic search + fosElastica + symfony2 + typehead + bloodhound + elastic suggesters

Share Button

(POST in BETA version, but I prefer release it, and enhance it later, than never publish ^^)

I want to use Elastic, to manage my search functionality.

It’s a symfony2 project, so I start my research by the integration of Elastic in symfony2, and FosElastica came up.
Thanks to FOS #friendOfSymfony2 people again, and again, your work is amazing !
Also nothing happends here without Elastica

This bundle provides integration with ElasticSearch and Elastica with Symfony2. Features include:

Integrates the Elastica library into a Symfony2 environment
Automatically generate mappings using a serializer
Listeners for Doctrine events for automatic indexing

I assume here, you have already a running symfony2 project, with data on it, I use this setup, to search over user, by name, mail, and over another data named « tags », a kind of internal references to group items.

Steps :

  1. install Elastic
  2. configure Elastic
  3. install FosElastica
  4. configure FosElastica
  5. try with Elastic web interface to confirm 4 firsts sets
  6. add typeahead.js to your page
  7. configure and implement bloodhound and typeahead
  8. Enjoy !

 

1 – Install of elasticsearch :

apt-get install elasticsearch

(If you are under ubuntu use sudo if you not root, tested on ubuntu 15.10 and Debian 8.)

To have elastic working, I’ve to :

chmod -R elasticsearch:elasticsearch /var/lib/elasticsearch

Both debian and ubuntu.

After that, you need to go to :

cd /etc/elasticsearch/

and open « elasticsearch.yml » to configure it.

2 – Configuration

(TODO : CORS config + security)

https://www.elastic.co/blog/found-elasticsearch-security

https://xenforo.com/community/threads/how-to-basic-elasticsearch-installation-debian-ubuntu.26163/

3 – Install FosElastica

with composer :

php composer.phar require friendsofsymfony/elastica-bundle "~3.0"

And enable it :

<?php
// app/AppKernel.php

public function registerBundles()
{
    $bundles = array(
        // ...
        new FOS\ElasticaBundle\FOSElasticaBundle(),
    );
}

4 - FosElastica configuration :

http://elasticsearch-users.115913.n3.nabble.com/Http-Cors-Setting-td4066032.html

https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-http.html#_settings_3

 

http://twitter.github.io/typeahead.js/examples/
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters-completion.html
https://github.com/FriendsOfSymfony/FOSElasticaBundle/issues/380
https://github.com/FriendsOfSymfony/FOSElasticaBundle/issues/445
https://github.com/FriendsOfSymfony/FOSElasticaBundle/issues/469
https://github.com/FriendsOfSymfony/FOSElasticaBundle/issues/673
https://github.com/FriendsOfSymfony/FOSElasticaBundle
http://stackoverflow.com/questions/21530063/how-do-we-set-remote-in-typeahead-js-0-10/21533204#21533204

Faster Autocomplete with typeahead.js, Elasticsearch & nginx


http://mycodde.blogspot.se/2014/12/typeaheadjs-autocomplete-suggestion.html
https://www.elastic.co/guide/en/elasticsearch/reference/master/search-suggesters.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-suggesters-completion.html

Share Button

[php] How to manage uploaded images in secure way

Share Button

I dev lately on an open source project, GRR (https://github.com/JeromeDevome/GRR/), it’s an old resources management script.

Writen a looooong time ago, « old school » code, but functional and pretty complete.

A security researcher , « kmkz » (Bourbon Jean-marie) – twitter: @kmkz_security, has discovered a security flaw which permit to upload a shell using the upload logo functionality.
Just need to rename the image like EvilShell.php.jpg …
The old code, never check if the submitted file have more than one extension, or if it’s an actual image… Just check if the final extension is jpg, gif or png.

After lot of reading about mime type, EXIF code injection, magic bytes, and million of stackoverflow questions, my little function :

https://github.com/nicolas-san/upload

This is a small, not POO, simple function to check :

  • multiple extension (logo.php.png)
  • extension allowed (jpg, png or gif)
  • good mime type with php finfo_file method http://php.net/manual/en/intro.fileinfo.php
  • if image is valid and remove EXIF with one of the imagecreateFORMAT function from GD http://php.net/manual/en/function.imagecreatefrompng.php
  • if target directory is writable
  • if move_uploaded_file pass without error

The idea is to start test with the obvious attempts, remove the EXIF data to be sure and rename the file to avoid long and wired names.

I think I cover all attack vectors with an image, but if you want test it, break it, improve or comment, do it !

<?php

/**
 * @author Bouteillier Nicolas <contact@kaizendo.fr>
 * Date: 13/01/16
 *
 * @license MIT
 *
 * @Param string $varName Name of the form input var
 *
 * @Param string $directoryImagePath Path of the directory where to save the image ex: '../images/' - The final / is needed
 * 
 * @Return string|int $pictureName string name of the new image or error code
 *
 * Error codes are :
 * -1 if varName ! isset
 * -2 if more than one extension ex: image.php.jpg
 * -3 extension not allowed (other than jpg, gif and png)
 * -4 mime type not allowed (other than image/gif, image/jpeg,image/png )
 * -5 imagecreate* failed, image not valid
 * -6 $directoryImagePath is not writable
 * -7 move_uploaded_file error
 * -8 can't delete moved file (you have to check if the file is still there and delete it)
 * -9 can't save GD created image, or destroy the return of imagecreate*
 *
 */

function saveUploadedFile($varName, $directoryImagePath)
{
    if (isset($_FILES[$varName])) {
        $file = $_FILES[$varName];
        /* first test only to block more than one extension file */
        if (count(explode('.', $file['name'])) > 2) {
            return -2;
        } elseif (preg_match("`\.([^.]+)$`", $file['name'], $match)) {
            /* here file has just one extension */
            $ext = strtolower($match[1]);
            if ($ext != 'jpg' && $ext != 'png' && $ext != 'gif') {
                return -3;
            } else {
                /**
                 * extension is ok
                 * third test with fileinfo to get the mime type with magic bytes
                 */
                $finfo = finfo_open(FILEINFO_MIME_TYPE);
                $fileType = finfo_file($finfo, $file['tmp_name']);
                /* fourth test depends on the extension, imagecreate with gd, to check if the image is valid and remove all exif data to avoid code injection */
                switch ($fileType) {
                    case 'image/gif':
                        $logoRecreated = @imagecreatefromgif($file['tmp_name']);
                        /* fix for transparency */
                        imageAlphaBlending($logoRecreated, true);
                        imageSaveAlpha($logoRecreated, true);
                        $extSafe = 'gif';
                        break;
                    case 'image/jpeg':
                        $logoRecreated = @imagecreatefromjpeg($file['tmp_name']);
                        $extSafe = 'jpg';
                        break;
                    case 'image/png':
                        $logoRecreated = @imagecreatefrompng($file['tmp_name']);
                        /* fix for transparency */
                        imageAlphaBlending($logoRecreated, true);
                        imageSaveAlpha($logoRecreated, true);
                        $extSafe = 'png';
                        break;
                    default:
                        return -4;
                        break;
                }
                if (!$logoRecreated) {
                    /* imagecreate* failed, the image is not good */
                    return -5;
                } else {
                    /** 
                     * valid image, good mime type
                     * destination is writable ?
                    */
                    /* generate random failename */
                    $randName = md5(uniqid(rand(), true));
                    $pictureName = $randName.'.'.$extSafe;
                    $picturePath = $directoryImagePath.$pictureName;

                    if (is_writable($directoryImagePath)) {
                        /* usage of move_uploaded_file to check on more time if the file is good (is it too much ?) */
                        $moveUploadReturn = move_uploaded_file($file['tmp_name'], $picturePath);
                        if (!$moveUploadReturn) {
                            return -7;
                        } else {
                            /* move_uploaded_file return is ok, I delete the file, and use the GD created exif free file instead */
                            $unlinkReturn = unlink($picturePath);
                            if (!$unlinkReturn) {
                                return -8;
                            } else {
                                /* the file is deleted, saving the new image */
                                switch ($extSafe) {
                                    case 'gif':
                                        $retourSaveImage = imagegif($logoRecreated, $picturePath);
                                        break;
                                    case 'jpg':
                                        $retourSaveImage = imagejpeg($logoRecreated, $picturePath);
                                        break;
                                    case 'png':
                                        $retourSaveImage = imagepng($logoRecreated, $picturePath);
                                        break;
                                }
                                $retourDestroy = imagedestroy($logoRecreated);
                                if (!$retourSaveImage || !$retourDestroy) {
                                    return -9;
                                }
                                /*
                                 * All tests are passed, the function return the path of the new image
                                 * */
                                return $pictureName;
                            }
                        }
                    } else {
                        return -6;
                    }
                }
            }
        }
    } else {
        return -1;
    }
}

var_dump($_FILES);
echo "<br>";

echo saveUploadedFile('file', './');

 

Share Button