This article follows Foucault's notion of the social diagram to illustrate how current practices in the field of child and youth work are premised in constructions of “otherness” which were produced during the Enlightenment and colonial periods of European history. It argues that this historical practice of creating otherness is at the heart of many of the frustrations, which take place between those we label children and youth and those we call adults. It further contends that perpetuating such colonial patterns obscures the wisdom and knowledge produced by youth and adults when they think and create together. To remedy this it is proposed that adult youth workers must cease attempting to dominate and control young people. To do this, adult ways of knowing must be interrogated and dismantled from the inside out. No investigation of the characteristics of the young people as other will ever yield information about dismantling otherness. The dismantling of otherness needs to come through an exploration of adult's own “local memory.” Through this exploration we, as adults, can become visible and through that visibility we can possibly become accountable to the young people we engage and to ourselves and our communities.