Leave-One-SubGroup-Out (LOSGO)

This example shows how to make a Leave-One-SubGroup-Out.

Import librairies

from museotoolbox.cross_validation import LeaveOneSubGroupOut
from museotoolbox.processing import extract_ROI
from museotoolbox import datasets

Load HistoricalMap dataset

raster,vector = datasets.load_historical_data()
field = 'Class'
group = 'uniquefid'
X,y,s = extract_ROI(raster,vector,field,group)

Create CV

if n_splits is False (default), the number of splits will be the smallest number of subgroup of all labels.

valid_size = 0.5 # Means 50%
LOSGO = LeaveOneSubGroupOut(verbose=False,random_state=12) #

Note

Split is made to generate each fold

LOSGO.get_n_splits(X,y,s)
for tr,vl in LOSGO.split(X,y,s):
    print(tr.shape,vl.shape)

Out:

(9308,) (3339,)
(10873,) (1774,)

Differences with sklearn

Sklearn do not use subgroups (only groups), so no hierarchical dependances.

from sklearn.model_selection import LeaveOneGroupOut
LOGO = LeaveOneGroupOut()
for tr,vl in LOGO.split(X=X,y=y,groups=s):
    print(tr.shape,vl.shape)

Out:

(11166,) (1481,)
(11511,) (1136,)
(12125,) (522,)
(10323,) (2324,)
(12116,) (531,)
(11686,) (961,)
(12382,) (265,)
(11434,) (1213,)
(11781,) (866,)
(12111,) (536,)
(12262,) (385,)
(12641,) (6,)
(12507,) (140,)
(12645,) (2,)
(11711,) (936,)
(12149,) (498,)
(11802,) (845,)

Plot example

from __drawCVmethods import plotMethod
plotMethod('LOO-group')

Out:

/home/docs/checkouts/readthedocs.org/user_builds/museotoolbox/checkouts/v0.12/examples/cross_validation/__drawCVmethods.py:35: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance.  In a future version, a new instance will always be created and returned.  Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.
  ax = f.add_subplot(111)