When reading Matthew the first book of the Gospels one cannot possibly think of Joseph, Mary’s Husband as nothing but virtuous. As we read in verse 1: 18,
18 The birth of Jesus Christ came about this way: After His mother Mary had been betrothed to Joseph, it was discovered before they came together that she was pregnant by the Holy Spirit.
Notice the word “betrothed”; this was a known as a “binding”, the same as a modern marriage. So in other words it was discovered that Mary was pregnant before they “came together” aka “consummated”.
What a shock this was for them obviously as death by stoning was the punishment for this sort of adultery! We go on to read;
Matthew 1: 19, 19 So her husband Joseph, being a righteous man, and not wanting to disgrace her publicly, decided to send her away secretly.
“Righteous” means in this case that he was also merciful and followed the Hebrew Laws to the letter. “Sending her away” meant to divorce her thus sparing her from disgrace. However before this was allowed to happen Joseph was visited in a dream by an Angel of God, as indicated by the next two verses;
Matthew 1: 20-21 20 But when he had considered this, behold, an angel of the Lord appeared to him in a dream, saying, “Joseph, son of David, do not be afraid to take Mary as your wife; for the Child who has been conceived in her is of the Holy Spirit.21 She will bear a Son; and you shall call His name Jesus, for He will save His people from their sins.”
There is no scripture following that ever suggests that Joseph doubted Mary or the Angel of God! Not only was there no doubt, he obeyed the Angel and went ahead with plans of marrying her! He then took her as his wife and did not have sexual relations with her until after Jesus was born (as a result Mary bore more children to Joseph, something the Roman Catholics disagree with). In Rabbinic tradition much later Mary was charged with adultery, however the fact that Joseph married her would indicate he did not believe this to be true.