If half of my family originated from the Sahara desert why are they considered sub saharan african?

Geographically Sub Sahara refers to the areas south of Sahara. However in a social or political context it is used in as the area of Africa that is not North Africa. While North Africa is considered part of the Arab world, Sub Saharan African is sometimes called 'Black Africa'.