A man accused of raping a beloved 82-year-old grandmother more than four decades ago has faced court for the first time.