Fraska Portal

Exploring the WebSphere Commerce world

Subscribe to this list via RSS Blog posts tagged in SOLR
WebSphere Commerce, the curious life of a front-end catalog request

The life begins in a random internet browser when an unaware user- browsing the Aurora shop- makes a simple request asking for details about the child laptop product. This request can be translated in a Uniform Resource Locator exactly as the following:

http://www.mywcsaurora.com/shop/en/aurora/eletronics/laptops/child-laptop

After just few milliseconds, or maybe few hours mostly depending from the performance of the Aurora shop, the URL assumes the shape of a page, the death of the request:

b2ap3_thumbnail_WCS_liferequest.png 

 

Between the request's birth and its death some many things have happened; I'll try to summarize the "typical life of a catalog request" in a WCS v7 FEP7 "world" marking just the communication "links" between the several components look after the request.

Posted by on in Administration
WebSphere Commerce, the SOLR extension index

One of the things I like most about the last enhancements- maybe introduced with FEP6, not sure of that- is the possibility to create an extension of the SOLR index. An extension index is, in a nutshell, an index which complements the data provided by the base index. A enlightening explanation is reported in the following IC page: Extending the WebSphere Commerce search base index schema.

According to the Info Center: "... dynamic information such as product inventory or ratings change more frequently. Therefore, it is put into a separate index, where it can be refreshed at different intervals than the base product index". That's awesome and it looks exactly the basic idea under the concept of Inventory Index (introduced with FEP6).

b2ap3_thumbnail_IC_SOLR_extension_index.png

Above is an extraction of the image reported in the IC page already mentioned before.

Using this feature we can finally think to implement re-index strategies with high frequency, pretty useful for dynamic data. The basic bricks used by the WCS team to build up this new feature are- according to my investigation- a custom Search Component (SolrSearchMultipleQueryComponent) and the possibility to make Join between different indexes.

That combination gives to the WCS implementation of SOLR an added value makes the SOLR native Join feature transparent to the developers. Let's review the main things ...

WebSphere Commerce, FEP7, Toolkit, Deploy SOLR Admin UI

The SOLR Admin UI is an interesting tool for monitoring the created cores, reviewing their configuration, querying them, understandng the cache performances, collecting useful troubleshooting information and many other things.

Having this tool in the hands of a System Admin could be really worth it. The UI is the following one:

WCS_SOLR_UI.png

The WCS InfoCenter describes the steps to follow in order to integrate the admin UI files in the solr-search web app: Info Center, Search performance tuning, Performance monitoring

However, it seems to have a couple of inaccuracies. So, I decided to report my experience deploying the SOLR Admin UI on FEP7 toolkit.

WebSphere Commerce, Review of the Search Navigation Flow

This time I prefer report this fundamental topic just with a schema I draw looking at the following IC pages:

It's the combination of the above documents; the following schema remarks the different interaction WCS runs in order to get Catalog information from SOLR.

I hope you can enjoy!

 

FIRST PART: from Request to Search Expression

SearchRequest_flow_1_20140708-081219_1.png

 

 

SECOND PART: from Expression to Search Response

SearchRequest_flow_2.png

THIRD PART: from Search Response to BOD noun

SearchRequest_flow_3.png

 

 

 

WebSphere Commerce, FEP 7, SOLR index pre-process error

Once the FEP7 installation is done and the starter store is published with some example data, the next step is the SOLR index configuration (setupSearchIndex), pre-process (di-preprocess) and build (di-buildindex).

However, during the pre-process step I faced an interesting exception. In fact, running the following command

di-preprocess.bat <WC_installfolder>\search\pre-processConfig\MC_<catalogId>\DB2

the process ended with the exception:

SEVERE: Exception message: CWFDIH0001: Data Import encountered a generic error. The error is null. , stack trace: com.ibm.commerce.foundation.dataimport.exception.DataImportSystemException: CWFDIH0001: Data Import encountered a generic error. The error is null. 
	at com.ibm.commerce.foundation.dataimport.preprocess.DataImportPreProcessorMain.processDataConfig(DataImportPreProcessorMain.java:957)
	at com.ibm.commerce.foundation.dataimport.preprocess.DataImportPreProcessorMain.execute(DataImportPreProcessorMain.java:802)
	at com.ibm.commerce.foundation.dataimport.preprocess.DataImportPreProcessorMain.main(DataImportPreProcessorMain.java:289)
Caused by: java.lang.NullPointerException
	at com.ibm.commerce.foundation.dataimport.preprocess.StaticAttributeDataPreProcessor.process(StaticAttributeDataPreProcessor.java:299)
	at com.ibm.commerce.foundation.dataimport.preprocess.DataImportPreProcessorMain.processDataConfig(DataImportPreProcessorMain.java:919)
	... 2 more
WebSphere Commerce, the "dark side" of the interactions between SOLR delta index and the DataLoad - Part 2

 

This article reports another example of interaction between SOLR delta index and data load process.

In particular, If we have to introduce the delta index- in place of the full- and the data load is already set up, or we have to plan from scratch the data load process, we should keep in mind that the data load operations, the order they are executed and the delta updates recorded in the temporary delta tables (TI_DELTA_CATENTRY and TI_DELTA_CATGROUP) are strictly connected; we should not consider the data load a black box respect to the SOLR index, when we need to update the system through the delta mode. The SOLR index and the Data Load are unfortunately tied in a way we cannot think to build a loading process without knowing in which way they interact.

I report two specific cases: 

  1. delete bundles (reported in the previous article);
  2. delete sales categories;
WebSphere Commerce, the "dark side" of the interactions between SOLR delta index and the DataLoad

This topic's title could seems a bit alarming but it's just a pinch of irony I would give, most of all because I struggled for so many weeks on this feature that it was like fighting against the "dark". However, I just want to focus on few things we should think about before to plan the introduction of the delta index when the environment is updated through the data load.

In particular, if we have to introduce the delta index- in place of the full- and the data load is already set up, or we have to plan from scratch the data load process, we should keep in mind that the data load operations, the order they are executed and the delta updates recorded in the temporary delta tables (TI_DELTA_CATENTRY and TI_DELTA_CATGROUP) are strictly connected; we should not consider the data load a black box respect to the SOLR index, when we need to update the system through the delta mode. The SOLR index and the Data Load are unfortunately tied in a way we cannot think to build a loading process without knowing in which way they interact.

Ok, let's leave fancy words! I will report two specific cases: 

  1. delete bundles;
  2. delete sales categories (second part of this article);
WebSphere Commerce, Data Load and SOLR Delta Index

One of the most annoying thing with the previous version of WCS (until de v7 FEP5) was the impossibility to perform a delta build of SOLR index after a data load process also for few line changes  ... basically a wast of time and resources.

However, FEP5 has finally introduced the possibility to make it works. In fact, new mediators have been implemented to load the data load changes in the temporary index tables and than execute successfully a Delta build reindex.

To test this interesting and very useful feature I realized a specific use case with Aurora on Windows platform, WCS Toolkit Enterprise v7 FEP6, DB2 and SOLR.

The test case:

  1. Create a simple CSV file with the introduction of a new product: "Fancy Laptop"
  2. Configure the data load (not using the new mediators) to load this product
  3. Run the data load
    • test: the product does not appear in the Front End
  4. Run a DELTA rebuild of the SOLR index
    • test: the product does not appear in the Front End
  5. Run a FULL rebuild of the SOLR index
    • test: the product appears in the Front End
  6. Update data load configuration (adding the use of the new mediators and properties)
  7. Change the product's name in the CSV ("New Fancy Laptop")
  8. Run the data load
  9. Run a DELTA rebuild of the SOLR index
    • test: the product appears in the Front End

Most Popular Post

WebSphere Commerce, the SOLR extension index
Administration
Rate this blog entry:
4
WebSphere Commerce, Data Load and SOLR Delta Index
Data Load
Rate this blog entry:
0
WebSphere Commerce, FEP 7, SOLR index pre-process error
Administration
Rate this blog entry:
0

Latest Blogs

WebSphere Commerce, CommandLevelAuthorizationCache
Cache
Rate this blog entry:
0
WebSphere Commerce v8, toolkit exception, ClassNotFound db2
Administration
Rate this blog entry:
1
WebSphere Commerce, ATP migration
Store
Rate this blog entry:
0
WebSphere Commerce, the curious life of a front-end catalog request
Core
Rate this blog entry:
5
WebSphere Commerce, Performance analysis of few European stores
Performance
Rate this blog entry:
0