Who rejects the literal fulfillment of Biblical prophecy? Religious liberals call Biblical prophecy imaginary. For example, Farrell Till, who has debated with Norman Geisler:
Prophecy fulfillment is a popular argument that bibliolaters rely on in trying to prove the divine inspiration of the Bible. They claim that the Bible is filled with recorded events that prophets foretold years and even centuries before they happened. They argue that there is no way to explain how these predictions could have been so accurately made except to conclude that the Holy Spirit enabled the prophets who uttered them to see into the future. In prophecy fulfillment, then, they see evidence of God’s direct involvement in the writing of the Bible.
A very simple flaw in the prophecy-fulfillment argument is that foreseeing the future doesn’t necessarily prove divine guidance. Psychics have existed in every generation, and some of them have demonstrated amazing abilities to predict future events. Their “powers,” although mystifying to those who witness them, are not usually considered divine in origin. If, then, Old Testament prophets did on occasions foresee the future (a questionable premise at best), perhaps they were merely the Nostradamuses and Edgar Cayces of their day. Why would it necessarily follow that they were divinely inspired? Even the Bible recognizes the possibility that uninspired prophets can sometimes accurately predict the future.
George Peters, in his classic, The Theocratic Kingdom, accused those of spiritualizing Biblical prophecy of the same rejection of the literal fulfillment of prophecy and that this allegorizing of prophecy provides fuel for the skeptics:
The prophecies referring to the Kingdom of God, as now interpreted by the large majority of Christians, afford the strongest leverage employed by unbelievers against Christianity. Unfortunately, unbelief is often logically correct. Thus, e.g., it eagerly points to the predictions pertaining to David’s Son, showing that, if language has any legitimate meaning, and words are adequate to express an idea, Literalism would end their uncertainty in interpretation of this point! They unmistakably predict the restoration of David’s throne and kingdom, etc., and then triumphantly declare that it was not realized (so Strauss, Baur, Renan, Parker, etc.). They mock the expectation of the Jews, of Simeon, the preaching of John, Jesus, and the disciples, the anticipation of the early Church, and hastily conclude, sustained by the present faith of the Church (excepting only a few), that they will never be fulfilled; and that, therefore, the prophecies, the foundation upon which the superstructure rests, are false, and of human concoction. The manner of meeting such objections is humiliating to the Word and Reason; for it discards the plain grammatical sense as unreliable, and, to save the credit of the Word, insists upon interpreting all such prophecies by adding to them under the claim of spiritual, a sense which is not contained in the language, but suits their religious system adopted. Unbelief is not slow in seizing the advantage thus given, gleefully pointing out how this introduced change makes the ancient faith an ignorant one, the early Church occupying a false position, and the Bible a book to which man adds any sense, under the plea of spiritual, that may be deemed necessary for its defense (George N. H. Peters, The Theocratic Kingdom. Grand Rapids: Kregel, 1952, 1:167-168).
Of course, Peters is referring to Amillennialists who practice a literal, historic/grammatical hermeneutic with first coming prophecies but flip-flop and allegorize Second coming prophecies. Just as the plain sense of language is necessary for the fulfillment of first coming prophecies, Jesus was born in Bethlehem according to Micah 5:2, why not be consistent in your hermeneutic and interpret literally Second coming prophecies like Isaiah 11 and Revelation 20:1-7?