WebSphere Commerce 7 | Solr Build Index or Server Failed to Start | java.lang.OutOfMemoryError: Java heap space | List of files generated

When you are running Solr Build Index with low memory heap or if you are not able to start the server, you might see the following error,
[10/1/17 22:14:07:840 COT] 00000117 SystemErr     R Exception in thread "Thread-54" java.lang.OutOfMemoryError: Java heap space
[10/1/17 22:14:07:841 COT] 00000117 SystemErr     R     at org.apache.lucene.index.FreqProxTermsWriterPerField$FreqProxPostingsArray.<init>(FreqProxTermsWriterPerField.java:261)
[10/1/17 22:14:07:841 COT] 00000117 SystemErr     R     at org.apache.lucene.index.FreqProxTermsWriterPerField$FreqProxPostingsArray.newInstance(FreqProxTermsWriterPerField.java:279)
[10/1/17 22:14:07:841 COT] 00000117 SystemErr     R     at org.apache.lucene.index.ParallelPostingsArray.grow(ParallelPostingsArray.java:48)
[10/1/17 22:14:07:841 COT] 00000117 SystemErr     R     at org.apache.lucene.index.TermsHashPerField$PostingsBytesStartArray.grow(TermsHashPerField.java:307)
[10/1/17 22:14:07:842 COT] 00000117 SystemErr     R     at org.apache.lucene.util.BytesRefHash.add(BytesRefHash.java:324)
[10/1/17 22:14:07:842 COT] 00000117 SystemErr     R     at org.apache.lucene.index.TermsHashPerField.add(TermsHashPerField.java:185)
[10/1/17 22:14:07:842 COT] 00000117 SystemErr     R     at org.apache.lucene.index.DocInverterPerField.processFields(DocInverterPerField.java:171)
[10/1/17 22:14:07:842 COT] 00000117 SystemErr     R     at org.apache.lucene.index.DocFieldProcessor.processDocument(DocFieldProcessor.java:248)
[10/1/17 22:14:07:842 COT] 00000117 SystemErr     R     at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:253)
[10/1/17 22:14:07:843 COT] 00000117 SystemErr     R     at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:453)
[10/1/17 22:14:07:843 COT] 00000117 SystemErr     R     at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1529)
[10/1/17 22:14:07:843 COT] 00000117 SystemErr     R     at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:236)
[10/1/17 22:14:07:843 COT] 00000117 SystemErr     R     at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:160)
[10/1/17 22:14:07:844 COT] 00000117 SystemErr     R     at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69)
[10/1/17 22:14:07:844 COT] 00000117 SystemErr     R     at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51)
[10/1/17 22:14:07:844 COT] 00000117 SystemErr     R     at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:729)
[10/1/17 22:14:07:844 COT] 00000117 SystemErr     R     at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:556)
[10/1/17 22:14:07:844 COT] 00000117 SystemErr     R     at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:100)
[10/1/17 22:14:07:845 COT] 00000117 SystemErr     R     at org.apache.solr.handler.dataimport.SolrWriter.upload(SolrWriter.java:70)
[10/1/17 22:14:07:845 COT] 00000117 SystemErr     R     at org.apache.solr.handler.dataimport.DataImportHandler$1.upload(DataImportHandler.java:235)
[10/1/17 22:14:07:845 COT] 00000117 SystemErr     R     at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:510)
[10/1/17 22:14:07:845 COT] 00000117 SystemErr     R     at org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:414)
[10/1/17 22:14:07:845 COT] 00000117 SystemErr     R     at org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:329)
[10/1/17 22:14:07:845 COT] 00000117 SystemErr     R     at org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:237)
[10/1/17 22:14:07:845 COT] 00000117 SystemErr     R     at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:411)
[10/1/17 22:14:07:846 COT] 00000117 SystemErr     R     at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
[10/1/17 22:14:07:846 COT] 00000117 SystemErr     R     at org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:464)

If this is the case then you should be able to see the Java Heap Dumps in one of the following locations,
C:\IBM\WCDE_ENT70\bin
C:\IBM\WCDE_ENT70\logs
C:\IBM\WCDE_ENT70\wasprofile\bin
C:\IBM\WCDE_ENT70\wasprofile\logs

Java Heap Dumps will be with the following naming conversions,
 - heapdump.20171001.215526.5444.0009.phd
 - javacore.20171001.215526.5444.0010.txt
 - Snap.20171001.215526.5444.0011.trc

Please stop the server, cleanup all these heap dumps, which will be taking amount of size. After the server restart your application should be working fine.

WebSphere Commerce 7 | Storefront Catalog failed to Load | javax.el.ELException: Error reading 'subCategories' | Resolution

Sometimes, you might have merged/added/removed few files from WC Project and from that onwards,
 - Store might be coming up but, it displays none of the catalogs and throws the following exception
 - Entries in STORECAT, STOREDEFCAT might be correct as well
 - If you add catalogId in the URL as one of the parameter, it works
 - If this is the case, few Out of Box files from WC project might be missing
 - I couldn't be able to find which Out of Box WC file was missing
 - But, after replacing with Default WC project again, it started working fine

[9/25/17 16:10:47:213 COT] 0000008c LoggingHelper E /GenericJSPPageError.jsp - Error reading 'subCategories' on type com.ibm.commerce.catalog.beans.CategoryDataBean
                                 javax.el.ELException: Error reading 'subCategories' on type com.ibm.commerce.catalog.beans.CategoryDataBean
    at javax.el.BeanELResolver.getValue(BeanELResolver.java:69)
    at javax.el.CompositeELResolver.getValue(CompositeELResolver.java:143)
    at org.apache.el.parser.AstValue.getValue(AstValue.java:102)
    at org.apache.el.parser.AstEmpty.getValue(AstEmpty.java:29)
    at org.apache.el.parser.AstNot.getValue(AstNot.java:26)
    at org.apache.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:263)
    at org.apache.jasper.runtime.PageContextImpl.proprietaryEvaluate(PageContextImpl.java:774)
    at com.ibm._jsp._FooterDisplayMenuCategories._jspx_meth_c_if_17(_FooterDisplayMenuCategories.java:7681)
    at com.ibm._jsp._FooterDisplayMenuCategories._jspService(_FooterDisplayMenuCategories.java:1123)
    at com.ibm.ws.jsp.runtime.HttpJspBase.service(HttpJspBase.java:99)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1667)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1602)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:113)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:80)
    at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:908)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:939)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:507)
    at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:181)
    at com.ibm.wsspi.webcontainer.servlet.GenericServletWrapper.handleRequest(GenericServletWrapper.java:121)
    at com.ibm.ws.jsp.webcontainerext.AbstractJSPExtensionServletWrapper.handleRequest(AbstractJSPExtensionServletWrapper.java:259)
    at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.include(WebAppRequestDispatcher.java:732)
    at org.apache.taglibs.standard.tag.common.core.ImportSupport.acquireString(ImportSupport.java:384)
    at org.apache.taglibs.standard.tag.common.core.ImportSupport.doEndTag(ImportSupport.java:220)
    at com.ibm._jsp._CachedHeaderDisplay._jspx_meth_c_import_5(_CachedHeaderDisplay.java:11806)
    at com.ibm._jsp._CachedHeaderDisplay._jspService(_CachedHeaderDisplay.java:1431)
    at com.ibm.ws.jsp.runtime.HttpJspBase.service(HttpJspBase.java:99)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:831)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1667)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1602)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:113)
    at com.ibm.ws.webcontainer.filter.WebAppFilterChain._doFilter(WebAppFilterChain.java:80)
    at com.ibm.ws.webcontainer.filter.WebAppFilterManager.doFilter(WebAppFilterManager.java:908)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:939)
    at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:507)
    at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.handleRequest(ServletWrapperImpl.java:181)
    at com.ibm.wsspi.webcontainer.servlet.GenericServletWrapper.handleRequest(GenericServletWrapper.java:121)
    at com.ibm.ws.jsp.webcontainerext.AbstractJSPExtensionServletWrapper.handleRequest(AbstractJSPExtensionServletWrapper.java:259)
    at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.include(WebAppRequestDispatcher.java:732)
    at org.apache.taglibs.standard.tag.common.core.ImportSupport.acquireString(ImportSupport.java:384)
    at org.apache.taglibs.standard.tag.common.core.ImportSupport.doEndTag(ImportSupport.java:220)
    at com.ibm._jsp._GenericError._jspx_meth_c_import_2(_GenericError.java:14380)

WebSphere Commerce 7 | trace/SystemOut/SystemErr Loggers | Special Characters in Logs

While debugging one of the issues, in the trace logs, most of Out of Box logs are getting printed with special characters,

This might be SSL logging and we need to turn off from WAS Console.

If this is the case, please validate whether any loggers are enabled in WAS Console, 
Application servers > Java and Process Management > Process Definition > Java Virtual Machine > Custom Properties

 
We have removed/disabled javax.net.debug and all the loggers are coming up fine.

WebSphere Commerce 7 | SoapUI | REST Calls over HTTPS | SSLHandshakeException

I tried to use SoapUI tool for testing my WebSphere Commerce REST Services. It is working fine with HTTP URLs and not the HTTPS URLS and it is showing the following exceptions in SoapUI,
Wed May 03 16:42:27 ART 2017:DEBUG:Attempt 1 to execute request
Wed May 03 16:42:27 ART 2017:DEBUG:Sending request: POST /wcs/resources/store/10151/guestidentity?updateCookies=true HTTP/1.1
Wed May 03 16:42:27 ART 2017:DEBUG:I/O error closing connection
   javax.net.ssl.SSLException: Connection has been shutdown: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
       at sun.security.ssl.SSLSocketImpl.checkEOF(Unknown Source)
       at sun.security.ssl.SSLSocketImpl.checkWrite(Unknown Source)
       at sun.security.ssl.AppOutputStream.write(Unknown Source)
    .......
    .......
   Caused by: java.io.EOFException: SSL peer shut down incorrectly
       at sun.security.ssl.InputRecord.read(Unknown Source)
       ... 27 more
Wed May 03 16:42:27 ART 2017:DEBUG:Closing the connection.
Wed May 03 16:42:27 ART 2017:DEBUG:Connection closed
Wed May 03 16:42:27 ART 2017:DEBUG:Connection shut down
Wed May 03 16:42:27 ART 2017:ERROR:Exception in request: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
Wed May 03 16:42:27 ART 2017:ERROR:An error occurred [Remote host closed connection during handshake], see error log for details
Wed May 03 16:42:27 ART 2017:INFO:Error getting response for [https://localhost.guestidentity:Request 1]; javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake.

And in my WebSphere Commerce logs, here is the error message,
[5/3/17 12:13:50:289 ART] 0000004e SSLHandshakeE E   SSLC0008E: Unable to initialize SSL connection.  Unauthorized access was denied or security settings have expired.  Exception is javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1 not enabled or not supported
    at com.ibm.jsse2.kb.z(kb.java:241)
    at com.ibm.jsse2.SSLEngineImpl.b(SSLEngineImpl.java:19)
    at com.ibm.jsse2.SSLEngineImpl.c(SSLEngineImpl.java:169)
    at com.ibm.jsse2.SSLEngineImpl.wrap(SSLEngineImpl.java:184)
    at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:5)
    at com.ibm.ws.ssl.channel.impl.SSLUtils.handleHandshake(SSLUtils.java:744)
    at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.readyInbound(SSLConnectionLink.java:566)
    at com.ibm.ws.ssl.channel.impl.SSLConnectionLink.ready(SSLConnectionLink.java:295)
    at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:214)
    at com.ibm.ws.tcp.channel.impl.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:113)
    at com.ibm.ws.tcp.channel.impl.AioReadCompletionListener.futureCompleted(AioReadCompletionListener.java:175)
    at com.ibm.io.async.AbstractAsyncFuture.invokeCallback(AbstractAsyncFuture.java:217)
    at com.ibm.io.async.AsyncChannelFuture.fireCompletionActions(AsyncChannelFuture.java:161)
    at com.ibm.io.async.AsyncFuture.completed(AsyncFuture.java:138)
    at com.ibm.io.async.ResultHandler.complete(ResultHandler.java:204)
    at com.ibm.io.async.ResultHandler.runEventProcessingLoop(ResultHandler.java:775)
    at com.ibm.io.async.ResultHandler$2.run(ResultHandler.java:905)
    at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1656)
Caused by: javax.net.ssl.SSLHandshakeException: Client requested protocol TLSv1 not enabled or not supported

Issue:
SoapUI is not enabled with TLS 1.2

Resolution to enabled the SoapUI with TLS 1.2,
1. Close the SoapUI
2. Open the following file, C:\Program Files\SmartBear\SoapUI-5.2.1\bin\SoapUI-5.2.1.vmoptions
3. Add the following lines at the end,
    -Dsun.security.ssl.allowUnsafeRenegotiation=true
    -Dsoapui.https.protocols=SSLv3,TLSv1.2

 
4. Open the SoapUI again and it should start working,

WebSphere Commerce 7 | REST | Basic Authentication | How to validate Basic Authentication from REST Browser Plugins & Applications?

If third party vendor is calling your REST Service then user details has provided in the following way,

a. Calling REST Service tools/plugins available with Browsers (Eg: REST Client)
    1. Click on Authentication

    2. Provide Username & Password in the Basic Authentication

    3. REST Client Plugin will automatically convert this to Encoded Credentials format,

b. Using Application,
    1. The user name and password are combined into a string called username:password. User names and passwords that contain a colon character (:) are not supported.
    2. The resulting string literal is then encoded using the RFC2045-MIME variant of Base64, except it is not limited to 76 characters per line.
    3. The authorization method and a space is then put before the encoded string. For example, Basic
        For example, if the user agent uses Aladdin as the user name and open sesame as the password, the header is formed in the following way:
        Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==

If you pass these credentials in these formats, automatically WebSphere Commerce will validate the user credentials from USERREG Tables and error will be thrown if the credentials are not valid.

WebSphere Commerce 7 | Solr | Preprocess Command failed to run | Queries to find the Data Issues

You might have noticed Solr Preprocess might have failed because of the Data issues in the Master Catalog.

Here are the few queries to check the Data Issues in the Master Catalog and making the Solr Preprocess to fail,

Query 1:  Same Products/SKUs has been assigned to Multiple Catalog Groups in the Master Catalog,
select catentry_id, (select partnumber from catentry where catentry_id = c.catentry_id) partnumber,
        catgroup_id, (select identifier from catgroup where catgroup_id = c.catgroup_id) catgroup_identifier
from catgpenrel c
where catentry_id in
        (select catentry_id
        from catgpenrel
        where catalog_id = 10001
        group by catentry_id
        having count(catentry_id) > 1
        )

Query 2:  One Category is assigned with multiple Categories in the Master Catalog
select catentry_id, catgroup_id
from catgpenrel
where catentry_id in
    (select catentry_id from catgpenrel where catalog_id = 10001
        group by catentry_id having count(catentry_id) > 1
    )

Query 3:  One SKU assigned to multiple Parent Products in the Master Catalog
select child_catentry_id, count(catentry_id_parent) parents_count from (
select c1.partnumber as parent_partnumber, c1.catentry_id as catentry_id_parent,
c2.partnumber as child_partnumber, c2.catentry_id as child_catentry_id, cgr.catalog_id
from catentry c1, catentry c2, catentrel cr, catgpenrel cgr
where c1.catentry_id = cr.catentry_id_parent
and c2.catentry_id = cr.catentry_id_child
and cgr.catentry_id = c1.catentry_id
and cgr.catalog_id = 10001
and c1.markfordelete = 0
and c1.buyable = 1)
group by child_catentry_id
having count(catentry_id_parent) > 1

Query 4:  SKU (Item) assigned as Child to one Parent Product and also marked as Parent to another Child
select catentry_id, partnumber, c2.catentry_id_child as markedasparent
from catentry c1, catentrel c2
where c1.catentry_id = c2.catentry_id_parent
and catenttype_id = 'ItemBean'

Query 5:  Product assigned as Child to another Product
select catentry_id, partnumber, c2.catentry_id_parent as markedaschild
from catentry c1, catentrel c2
where c1.catentry_id = c2.catentry_id_child
and catenttype_id = 'ProductBean'

WebSphere Commerce 7 | Management Center | Customer Segment Failed to Save | Error Code CMN0207E

When we tried to Create/Update Customer Segment, it is failing to save and throwing the following exception (With Error Code: CMN0207E),

[1/9/17 10:41:55:286 GMT] 00000d64 AbstractChang I com.ibm.commerce.foundation.server.command.bod.AbstractChangeBusinessObjectDocumentCmdImpl save(Map) The com.ibm.commerce.member.facade.server.commands.ChangeMemberGroupCmdImpl.save(Map) was not implemented and the default behavior does not invoke any logic.  It is recommend to implement this method.
[1/9/17 10:41:55:852 GMT] 00000d64 CommerceSrvr  A com.ibm.commerce.scheduler.commands.AddBroadcastJobCmdImpl validateParameters CMN0207E The value of the parameter "name" is not correct.
[1/9/17 10:41:55:852 GMT] 00000d64 ExtendedInfo  I   CWXFR9010I: Extended information : [componentId=com.ibm.commerce.member]
[1/9/17 10:41:55:853 GMT] 00000d64 ExtendedInfo  I   CWXFR9010I: Extended information : [componentId=com.ibm.commerce.member]
[1/9/17 10:41:55:855 GMT] 00000d64 ExtendedInfo  I   CWXFR9010I: Extended information : [componentId=com.ibm.commerce.member]
[1/9/17 10:41:55:858 GMT] 00000d64 ExtendedInfo  I   CWXFR9010I: Extended information : [componentId=com.ibm.commerce.member]
[1/9/17 10:41:55:853 GMT] 00000d64 BusinessObjec E   CMN0207E The value of the parameter "name" is not correct.

Correlation Identity: 61f647de:15974236cb5:-7a79
Additional Data:
        null
Current exception:
Message:
   _ERR_CMD_INVALID_PARAM
Stack trace:
com.ibm.commerce.exception.ECApplicationException: CMN0207E The value of the parameter "name" is not correct.
        at com.ibm.commerce.scheduler.commands.AddBroadcastJobCmdImpl.validateParameters(AddBroadcastJobCmdImpl.java:146)
        at com.ibm.commerce.command.ECCommandTarget.executeCommand(ECCommandTarget.java:135)
        at com.ibm.ws.cache.command.CommandCache.executeCommand(CommandCache.java:332)
        at com.ibm.websphere.command.CacheableCommandImpl.execute(CacheableCommandImpl.java:166)
Issue:
1. While saving any information in Management Center, it will pass "SiteAdminID", which has been mentioned in <Instance> tag of wc-server.xml. This value should be a valid user in the application

2. In our application, "wcsadmin" user is not available as we have deleted that user.
3. The application is trying to find this user and as it is available it is throwing the exception. And also, failed to save the same.
com.ibm.commerce.tools.segmentation.SegmentCache

Possible fixes:
 - Update wc-server.xml <Instance> tag with active user & password with wcsadmin access. Eg: siteadmin
 - Create the user which has been mentioned in wc-server.xml <Instance> tag with wcsadmin access

WebSphere Commerce 7 | IBM Developer Works | How to post questions in IBM Developer Works?

1. Login to IBM Developer Works
https://www.ibm.com/developerworks/
2. Search for "WebSphere Commerce" forum in the top right Search text box

3. In the resultant forums, click on the appropriate forum

4. Start the topic and provide details of your question & post of the same