---

See my Github page for the full source code described below.

Custom pre-rendering solution using PuppeteerSharp for a static website hosted on AWS S3 with source data stored in DynamoDB.

Getting Started

This solution is highly customized for a particular website. However, the concepts covered and the solution structure may provide value to others struggling with the same problems, namely search engine optimization for websites using early versions of Angular. These instructions will give you an idea of how to customize the solution for your own purposes.

Prerequisites

A Chrome executable, a website URL to be rendered, and either an existing sitemap.xml or content stored in AWS DynamoDB that can be used to create the sitemap. AWS credentials will be needed in the latter case.

Installing

Update the appsettings.json file with details of the prerequisites above. You can alter the Program.cs steps as needed for your own solution. For example, you can skip the steps for loading site content from DynamoDB to generate the sitemap if you already have one and only need to render.

Deployment

To run this as part of a CI/CD pipeline, you will need to host this console application on a machine also running Chrome. The Program.cs could be adapted to run on a continuous loop or to be triggered as needed, possibly as a scheduled task.

Built With

• .NET Core - The console application framework used.
• Puppeteer Sharp - Puppeteer Sharp is a .NET port of the official Node.JS Puppeteer API.
• AWS S3 - Amazon's Simple Storage Solution on which the AngularJS website is hosted.
• AWS DynamoDB - Amazon's managed NoSQL solution used to host the dynamic content for the static website.

Acknowledgments

The following links proved useful on the journey to creating this solution. They provided a guide as to what I did and did not want to do in my solution.

Eric Lu - SEO for AngularJS on S3

Prerender.io - Middleware you install on a server to check if requests come from a crawler.

Zanon - Provided an introduction to PhantomJS which ultimately led to Puppeteer.

StackOverflow - Thread discussed using Prepender.io in the context of AWS Lambda.

Code Sample

public static async Task<List<Route>> GetContent(List<Route> routes, bool exportHtml = false, AmazonS3Client s3Client = null)
        {
            try
            {
                var result = new List<Route>();
                var options = new LaunchOptions
                {
                    Headless = true,
                    ExecutablePath = ChromeExePath
                };
               
                using (var browser = await Puppeteer.LaunchAsync(options))
                {
                    foreach (var route in routes)
                    {
                        Console.WriteLine($"Loading page: {route.Path}");

                        using (var page = await browser.NewPageAsync())
                        {
                            var response = await page.GoToAsync(route.Path, new NavigationOptions
                            {
                                WaitUntil = new WaitUntilNavigation[]
                                {
                                    WaitUntilNavigation.Networkidle0
                                }
                            });

                            route.Content = await page.GetContentAsync();
                            
                            if (!string.IsNullOrEmpty(route.Content))
                            {
                                route.Content = Format(route.Content);
                            }

                            result.Add(route);

                            if (exportHtml && !string.IsNullOrEmpty(route.Content) && 
                                route.Uri != null && route.Query != "//" && !route.Query.Contains("sitemap"))
                            {
                                Console.WriteLine($"Exporting page: {route.Path}");
                                await FileManager.Export(route);
                            }
                        }
                    }
                }

                return result;
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Render error: {ex.Message}. {ex.StackTrace}.");
                return null;
            }
        }