Many examples of how "Context is king," including the one just given, concern the history of ideas. If you want to understand, for example, the various conceptions of "the messiah" in the New Testament documents, then you need to know something of how the idea, messiah, functioned in the world in which the NT was written. So the Dead Sea Scrolls, Josephus, pseudepigraphical texts (esp. 1 Enoch and 4 Baruch), and other texts come into play.
But context isn't simply about ideas. The historical figures who interest us—Paul, Peter, Herod, and even (or especially) Jesus—lived in real time and real space, handled real objects, engaged in real social interactions with other real people, and so on. And the realness, or the materiality, of the world these people lived in can have important consequences for our own efforts to understand ancient texts and reconstruction historical scenarios.
So I especially enjoyed the following paragraph, from Pieter J. J. Botha's essay, "'Publishing' a Gospel: Notes on Historical Constraints to Gospel Criticism" (pp. 335–52 in The Interface of Orality and Writing: Speaking, Seeing, Writing in the Shaping of New Genres [A. Weissenrieder and R. Coote, eds.; WUNT 260; Tübingen: Mohr Siebeck, 2010]):
Readers, audiences and authors are products of particular historical conditions. By studying the formats of books and their typographical disposition(s) we can learn how they were created, how they were read, and undersatnd something about their possible meanings. In writing and reading materials, technology and human activity interact, and these various specifics must be taken into account when written artifacts are to be understood. Historical investigation requires that textual criticism, the historical study of books and cultural history be interrelated to describe "the variations that differentiate the 'readable space' (the texts in their material and discursive forms) and those which govern the circumstances of the 'actualization' (the readings seen as concrete practices and interpretive procedures)." (336, citing R. Chartier, "Laborers and Voyagers: From the Text to the Reader," Diacritics 22 : 50)