I believe the film is an allegory of Scott's quest to make the greatest film ever.
I believe the black goo was just a poison device so that an actor could die on screen and flex their death scene muscle. The engineers are how Scott visualises great actors, which is why they are almost human but with hollywood physiques and his vision of earth is a paradise full of astonishing actors. The engineer at the start completely sold his death sequence by dissolving down to the molecular level, which we can all agree is an impressive commitment to the arts.
I think David represented the director in his quest to rally decent acting performances out of the team and make the best film ever by revisiting the past glories and using the techniques of their predecessors to improve the current job lot of acting - consider how when David poisoned Charlie he started to improve in acting skill which was demonstrated when he became an engineer.
David ensured that the main woman, possibly the worst actress in all known space, saw the acting crash course by the disembodied head pulling off a wide range of emotions using just facial expressions. This was an attempt to demonstrate what he was after, however the main woman, unable to cope with such acting genius and knowing that she would never attain such heights, tried to freeze the head mid performance, but the power of its acting overcame technology and he literally exploded with talent. As all earlier attempts had failed, David tried to impregnate her with the genetically engineered offspring of the engineers, hoping that if he could not get good acting out of her, then maybe he could at least get a good actor out of her.
David as the director is also reflected in him being cast as a robot, which is why he has no demonstrable expression as being the director and a professional he has to keep detached from the performance. Realising the movie is a bust he sets out to find the last great actor, whom he wakes with a musical introduction and speaks to him in the language of the stars, seeking affirmation that he is a great director making a great film, bringing entertainment to the plebians from the great acting gods, ala prometheus. The last great actor, so horrified by the terrible makeup applied to the old man and further infuriated by the talentless hacks in front of him realises that there is no hope for acting or the movie industry, so single handedly tries to kill the movie in its tracks by removing the protagonists. Sadly, in a fit of irony, he is enveloped in the cloying tentacles of the CGI monster, both literally in the film and figuratively as CGI further replaces real people of talent in favour of computer generated representations.
Vickers as the only decent actor amongst them unfortunately dies by being unable to avoid the linear shackles of terrible writing and is killed by a spaceship shaped like a giant croissant representing those eaten by the script writers when they cobbled it together over coffee. We then see how the main woman avoids this fate, as owing to being the worst actress in space she cannot convey what is written down anyway, so is essentially a law unto her self and so sidesteps the writers intentions of death. She meets up with the leftover ego of the director, who is now just a head to demonstrate that to make a great film you don't need a body, just vision - and together they head off to seek revenge against the planet where great actors are born and make them feel so small and inadequate in space, so somewhere in space it is safe to assume an acting academy is about to get messed up.