Leave-One-SubGroup-Out (LOSGO)

This example shows how to make a Leave-One-SubGroup-Out.

Import librairies

from museotoolbox.cross_validation import LeaveOneSubGroupOut
from museotoolbox.processing import extract_ROI
from museotoolbox import datasets

Load HistoricalMap dataset

raster,vector = datasets.load_historical_data()
field = 'Class'
group = 'uniquefid'
X,y,s = extract_ROI(raster,vector,field,group)

Create CV

if n_splits is False (default), the number of splits will be the smallest number of subgroup of all labels.

valid_size = 0.5 # Means 50%
LOSGO = LeaveOneSubGroupOut(verbose=False,random_state=12) #

Note

Split is made to generate each fold

LOSGO.get_n_splits(X,y,s)
for tr,vl in LOSGO.split(X,y,s):
    print(tr.shape,vl.shape)

Out:

(9324,) (3333,)
(10870,) (1787,)

Differences with sklearn

Sklearn do not use subgroups (only groups), so no hierarchical dependances.

from sklearn.model_selection import LeaveOneGroupOut
LOGO = LeaveOneGroupOut()
for tr,vl in LOGO.split(X=X,y=y,groups=s):
    print(tr.shape,vl.shape)

Out:

(11178,) (1479,)
(11519,) (1138,)
(12134,) (523,)
(10333,) (2324,)
(12128,) (529,)
(11697,) (960,)
(12389,) (268,)
(11441,) (1216,)
(11793,) (864,)
(12118,) (539,)
(12272,) (385,)
(12651,) (6,)
(12517,) (140,)
(12654,) (3,)
(11723,) (934,)
(12161,) (496,)
(11804,) (853,)

Plot example

from __drawCVmethods import plotMethod
plotMethod('LOO-group')

Out:

/home/docs/checkouts/readthedocs.org/user_builds/museotoolbox/checkouts/latest/examples/cross_validation/__drawCVmethods.py:35: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance.  In a future version, a new instance will always be created and returned.  Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.
  ax = f.add_subplot(111)