Skip to content

Grid change notes

Issue(s)

As of early 2020, the vue-bootstrap toolkit is used as the basis for most of the IV UI work. However, the b-table component is slow. The more information that is pushed in, the slower it gets - it’s not optimized to deal with thousands or tens of thousands of records.

The other aspect of the slowness is the data retrieval from the server. Some indexes help speed up some of the SQL, but the ‘conversion to JSON’ process (using the base Laravel ‘resources’ pattern) - is slow.
Part of the reason for its slowness is that the current process will do a lot of lazy loading on the shifts (for site/location/employee/etc).
Initial tests modifying this to a more eager-loading approach (and ensuring some indexes in place) cut data generation time moderately.

However, the Eloquent overhead still becomes noticeable on large amounts of data. A different approach was taken - to do a more-direct-to-SQL approach, using just the Laravel query builder.

Local testing on laptop showed 50000 rows of data being retrieved in around 3 seconds, with an additional 2 seconds of local parsing overhead (then passed to ag-grid). Based on other earlier comparison factors, local development speeds are roughly 2x faster than the current production system, meaning this 50000 rows examaple might take closer to 8-10 seconds.

Ag-grid

Reviewed some existing grid that can interact with VueJS (as that’s what the majority of the current IV code is written in), and ag-grid http://ag-grid.com fits the bill well, with a combination of base functionality in the free/community edition, with options for more functionality exposed in an enterprise version should the project get to the point where we’d need that. “Export to Excel” is probably the one ‘enterprise’ option which might be nice to have now, but there is an ‘export to CSV’ in the community edition, should we wish to add that in.

A sample ag-grid/vue combo is included, and we should look to migrate all existing grid usage over to ag-grid.
Search/filtering happens against each column, which in itself is likely providing a speed boost, but ag-grid itself is a rather mature base to being with.

Coding

There’s more boilerplate work to use it - at least, that’s my initial impression (I’m not an expert in this). Each column needs to define a value getter and a cell renderer. Well, each column that has filtering and that doesn’t use a standard text renderer, at any rate.

:valueGetter="siteValueGetter"
:width="200"
filter="agTextColumnFilter"
filterParams="filterParams: { resetButton: true, clearButton: true, debounceMs: 100, }"
:cellRenderer="siteCellRender"

....

siteValueGetter (params) {
    return params.data.site.name
},
siteCellRender (params) {
    return "<a href='/indevets/sites/"+params.data.site.id+"'>"+params.data.site.name+"</a>";
},

The filtering needs a ‘value getter’ to get the value to search against, and when custom cell renderers are in place, the default field value is not grabbed (which sort of makes sense - if it’s a complex object, what would the ‘value’ to search against be?)

note on client side vs server side

Both options provide some benefits, and some drawbacks. For this round, client side approach was chosen to provide an easier way of handling desired functionality (pagination and filtering, mainly).
Keeping track of local criteria (which page, which filters to use) to then send to the server adds additional development complexity for the sake of some local perception of speed.

There is a limit to client-side processing - if the loading and parsing of data for a local grid gets to a point where this is a noticeable and negative experience, a conversion of various tables to use server-side pagination/filtering may be required.

Client side

Pros

  • Pagination is all local - no round trips to server - faster experience
  • Easier column-level filtering

Cons

  • Longer initial load time
  • Eventual memory cap on how much data can be handled locally (tested with 50000 with acceptable load time)

Server side

Pros

  • No practical memory limits on datasets
  • Faster initial load time

Cons

  • Increased complexity to keep track of local state (filters, pagingation, column sort directions)
  • Impact on some local grid features (slower pagination, slower sorting, potential loss of exports, etc)