
.. DO NOT EDIT.
.. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY.
.. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE:
.. "auto_examples/features_detection/plot_fisher_vector.py"
.. LINE NUMBERS ARE GIVEN BELOW.

.. only:: html

    .. note::
        :class: sphx-glr-download-link-note

        :ref:`Go to the end <sphx_glr_download_auto_examples_features_detection_plot_fisher_vector.py>`
        to download the full example code.

.. rst-class:: sphx-glr-example-title

.. _sphx_glr_auto_examples_features_detection_plot_fisher_vector.py:


===============================
Fisher vector feature encoding
===============================

A Fisher vector is an image feature encoding and quantization technique that
can be seen as a soft or probabilistic version of the popular
bag-of-visual-words or VLAD algorithms. Images are modelled using a visual
vocabulary which is estimated using a K-mode Gaussian mixture model trained on
low-level image features such as SIFT or ORB descriptors. The Fisher vector
itself is a concatenation of the gradients of the Gaussian mixture model (GMM)
with respect to its parameters - mixture weights, means, and covariance
matrices.

In this example, we compute Fisher vectors for the digits dataset in
scikit-learn, and train a classifier on these representations.

Please note that scikit-learn is required to run this example.

.. GENERATED FROM PYTHON SOURCE LINES 20-78



.. image-sg:: /auto_examples/features_detection/images/sphx_glr_plot_fisher_vector_001.png
   :alt: plot fisher vector
   :srcset: /auto_examples/features_detection/images/sphx_glr_plot_fisher_vector_001.png
   :class: sphx-glr-single-img


.. rst-class:: sphx-glr-script-out

 .. code-block:: none

    /usr/lib/python3/dist-packages/sklearn/mixture/_base.py:275: ConvergenceWarning:

    Best performing initialization did not converge. Try different init parameters, or increase max_iter, tol, or check for degenerate data.

                  precision    recall  f1-score   support

               0       0.67      0.89      0.76        36
               1       0.71      0.78      0.74        41
               2       0.67      0.77      0.72        44
               3       0.68      0.49      0.57        51
               4       0.67      0.60      0.63        52
               5       0.57      0.57      0.57        47
               6       0.55      0.60      0.58        43
               7       0.57      0.69      0.63        45
               8       0.62      0.57      0.60        40
               9       0.74      0.55      0.63        51

        accuracy                           0.64       450
       macro avg       0.65      0.65      0.64       450
    weighted avg       0.65      0.64      0.64       450







|

.. code-block:: Python


    from matplotlib import pyplot as plt
    import numpy as np
    from sklearn.datasets import load_digits
    from sklearn.metrics import classification_report, ConfusionMatrixDisplay
    from sklearn.model_selection import train_test_split
    from sklearn.svm import LinearSVC

    from skimage.transform import resize
    from skimage.feature import fisher_vector, ORB, learn_gmm


    data = load_digits()
    images = data.images
    targets = data.target

    # Resize images so that ORB detects interest points for all images
    images = np.array([resize(image, (80, 80)) for image in images])

    # Compute ORB descriptors for each image
    descriptors = []
    for image in images:
        detector_extractor = ORB(n_keypoints=5, harris_k=0.01)
        detector_extractor.detect_and_extract(image)
        descriptors.append(detector_extractor.descriptors.astype('float32'))

    # Split the data into training and testing subsets
    train_descriptors, test_descriptors, train_targets, test_targets = train_test_split(
        descriptors, targets
    )

    # Train a K-mode GMM
    k = 16
    gmm = learn_gmm(train_descriptors, n_modes=k)

    # Compute the Fisher vectors
    training_fvs = np.array(
        [fisher_vector(descriptor_mat, gmm) for descriptor_mat in train_descriptors]
    )

    testing_fvs = np.array(
        [fisher_vector(descriptor_mat, gmm) for descriptor_mat in test_descriptors]
    )

    svm = LinearSVC().fit(training_fvs, train_targets)

    predictions = svm.predict(testing_fvs)

    print(classification_report(test_targets, predictions))

    ConfusionMatrixDisplay.from_estimator(
        svm,
        testing_fvs,
        test_targets,
        cmap=plt.cm.Blues,
    )

    plt.show()


.. rst-class:: sphx-glr-timing

   **Total running time of the script:** (0 minutes 36.666 seconds)


.. _sphx_glr_download_auto_examples_features_detection_plot_fisher_vector.py:

.. only:: html

  .. container:: sphx-glr-footer sphx-glr-footer-example

    .. container:: sphx-glr-download sphx-glr-download-jupyter

      :download:`Download Jupyter notebook: plot_fisher_vector.ipynb <plot_fisher_vector.ipynb>`

    .. container:: sphx-glr-download sphx-glr-download-python

      :download:`Download Python source code: plot_fisher_vector.py <plot_fisher_vector.py>`

    .. container:: sphx-glr-download sphx-glr-download-zip

      :download:`Download zipped: plot_fisher_vector.zip <plot_fisher_vector.zip>`


.. only:: html

 .. rst-class:: sphx-glr-signature

    `Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
