SFDC(Salesforce.com) Spark connector-(as sink/source)-scala
If there is a usecase to add SFDC as a connector to a Spark application (written in scala), then here you can refer below code (Test.scala) which will help you to read from SFDC and writing to this SFDC locally (provided you have been given with required connection details)
- First you need to connect to SFDC site using below info.
i. User name ii. Password iii.SFDC url/website/Lighting site/endpoint
2. Once you loging, you have to locate the existing objects/SfObjects/tables by https://xxx-deved.lightning.force.com/lightning/setup/ObjectManager/home
3. Then click on Developer console(new window pops up)inside Setup gear icon where you can query any Object using SOQL(Salesforce Object Query Language) through the query editor and execute and see the results there itself.
4. Use the Salesforce Object Query Language (SOQL) to search your organization’s Salesforce data for specific information. SOQL is similar to the SELECT statement in the widely used Structured Query Language (SQL) but is designed specifically for Salesforce data.
5. You have fetch data first from any Object and then read that data as a Dataframe in Spark.
6. If its not IP restricted then you don’t need security token- its just user name and password otherwise its required to have security token for the user(as logged into SFDC) and this needs to be suffixed with password while making a connection during read operation.
7. And in build.sbt, add the below library.(it depends on Scala version used in your application). For example,
libraryDependencies += "com.springml" % "spark-salesforce_2.11" % "1.1.3"8. Create a test.scala file, run it locally using the above and verify if you are able to connect to Snowflake and do read/write operations.
This is written to do a quick connection test from your local environement to SFDC from Spark with scala. Hope this helps!