Author: devin

LG LFX25961AL refrigerator woes

Moved into a house with a LFX25961AL fridge. Worked for about 2 years. The icemaker didn’t make a lot of ice, and the water flow was slow. Then it stopped cooling completely…

No cooling

Fridge and freezer stopped cooling completely. LG compressors aren’t known for their reliability, but this fridge is probably 15 years old.

After dismantling just about everything and using an IR thermometer to check coils, determined it was definitely the compressor.

This compressor users R134a refrigerant (the same as my car) so I figured I could give fixing it a try. Very likely the refrigerant had leaked out. We can confirm that with a few parts.

Warning: it may be illegal to service refrigerant yourself!

It’s crazy how many people online said not to do this…maybe they are all refrigerator techs!

Bought some parts, this kit includes everything you need to refill a fridge: https://www.amazon.com/gp/product/B08B43ZBTW/

Except the gas, which you can pick up pretty much anywhere, including Home Depot. Just get R134A refrigerant with leak sealant.

So first you’ll need to put the bullet valve on the end of the compressor (the not black part in the photo). This is there for a reason…to refill the compressor! It is the “low side” service tube. Follow the instructions to get it hooked up like this. Leave it closed, of course.

Then hook up the rest:

Leave the 134a valve closed, and open the bullet valve completely. If your fridge isn’t on then turn it on now and wait a while.

From what I read, the low side reading should be around 0-3 PSI if the compressor is working correctly.

Since my refrigerant was empty, I saw a reading of -3 PSI. This means that the compressor is pulling nothing through the system.

What you have to do then is slowly start adding R134a until the pressure stabilizes positive. Refill a little bit, stop, refill a little bit, stop…the needle will start inching its way up with every burst.

Eventually the PSI stabilized to around 2-3 and I closed the bullet valve and disconnected everything.

I could already tell it was starting to work, because negative temps were blowing out of the freezer compartment.

Several hours later:

Mission accomplished! Before I boarded everything back up, I was unable to see any of the leak sealant, which is dyed red. It must have been an extremely minor crack. This was actually 2 years ago and has been working fine for a while.

Slow ice, slow water

Alright so obviously this is water pressure. But it wasn’t any of the lines and it wasn’t the water filter. The ice tray wouldn’t fill up all the way and water would pour out really slowly.

It was the bullet valve installed downstairs which probably corroded over. Also those are sometimes illegal due to local codes because of how often they fail.

Turned off the bullet valve (I’ll replace it with a sharkbite later) and capped the end just in case.

Got some water line and ran it from the sink line instead, with this: https://www.amazon.com/gp/product/B01KIJZE0S

Fixed. Now makes a lot of ice and water dispenses much faster.

Water leak!

Uh oh!. Water leak behind the fridge.

Water had pooled up near the evaporator pool in the back of the fridge and started to overflow. I dried everything up and let it run a little bit. Water droplets were coming out of the water line going up to the fridge.

Looking closer, there was electrical tape around where a small hole was in the line. Previous owner must have had a small leak and electrical tape fixed it enough, due to the low pressure from before. When I fixed it, the high pressure caused more of a leak.

Easy fix requiring a scissor: remove the clamp, pull the line, cut the line, put it back in. Problem solved.

Now it’s too cold

Years later after the compressor fix, the refrigerator compartment froze over. My thought was it was the sensor at the top of the fridge which was possibly broken. The wire had been bent, so maybe it shorted. The level of resistance indicates temperature so that was my guess. The fridge was not throwing any errors.

I bought a new thermistor but it seems absolutely impossible to replace without taking apart the entire fridge. Unlike other fridge models, this one can’t be unplugged. The replacement part had bare leads and a 10 foot lead.

At some point in diagnosing, either playing with the cable or rebooting the fridge, it went back to normal and has been fine for months. Still unsure what the problem was. Possible it could have been the flap between the fridge and freezer, but I was unable to confirm.

No water, no ice for you

The ice and water dispensers stopped working completely. When I rebooted the fridge, the icemaker lights would flash. When trying to dispense water, the dispenser LED would also flash. After about 15 seconds, nothing would respond. To me it looked like a capacitor issue I encountered on another wine fridge, where not enough power was making it out of the mainboard. Manually turning the LED on or off through the display worked fine.

Just to make sure, I removed the icemaker and water solenoid just in case one of them was shorting. But the problem seemed the same, the dispenser LED would flash and then not respond after a while.

I also thought it could be the relay board that sits behind the dispenser panel. That controls the power given to the grind motor and the water solenoid. However there are no capacitors on that board and it looks like at least some power was making it to the ice maker and dispenser, since I saw LEDs light up. I also don’t think the ice maker is powered through that board.

Looking at the schematics in the service manual: https://www.manualslib.com/manual/749298/Lg-Lfx25961-Series.html

On the main assembly it says CON2 goes directly to the icemaker assembly. So if both the icemaker and the dispenser are not working, I think that’s a good indication that the problem is in the board somewhere.

Previously, I had encountered an error which indicated the mainboard was bad, but after rebooting the fridge, it went away. Perhaps this is an issue now.

The mainboard was $28 on ebay so I figured it was worth a shot without investigating further.

The culprit!

After tons of testing and looking at wiring diagrams that were just…wrong, I finally got time to get out the multimeter and test every lead from the top of the fridge (in the door) to every connector in the freezer.

Well, unfortunately I found that the neutral wire wasn’t connecting to the top of the fridge. No obvious damage anywhere that I could see. Probably in the door, which would be impossible to repair.

So instead, I confirmed that the blue wire for the ice maker and the relay board were neutral, and power was not getting to the relay board.

With basically no other option, I ran a new neutral from the top of the fridge to the relay board. It’s not pretty but…everything works.

improve AWS EFS sparse file throughput with fpsync

After running into an issue when copying thousands of files at one to EFS, I came across https://aws.amazon.com/premiumsupport/knowledge-center/efs-troubleshoot-slow-performance/

Let’s look at some benchmarks – the issue was that this job was taking 30+ seconds which would have timed out the HTTP server and it’s not yet a background job. It unzips a file to a temporary directory (not in EFS), does some validation, then copies the contents to EFS. The zip in question was 3000 files, around ~150MB extracted.

# find /big_dir | wc -l
3098
# time cp -R /big_dir /efs_dir

real    0m37.957s
user    0m0.052s
sys     0m0.960s

Ouch, that’s not good. We could try rsync, maybe that will help:

rsync -r /big_dir /efs_dir

real    1m10.210s
user    0m0.931s
sys     0m1.744s

Even longer! Why is this? It’s because:

Metadata I/O occurs if your application performs metadata-intensive, operations such as, "ls," "rm," "mkdir," "rmdir," "lookup," "getattr," or "setattr", and so on. Any operation that requires the system to fetch for the address of a specific block is considered to be a metadata-intensive workload.

rsync is also checking the destination file to see if it needs to sync it, which causes a bottleneck. So plain rsync and cp aren’t an option.

The issue is that Elastic File System is not built for serial operations. That is, copying a file, waiting, and copying the next one. EFS must replicate all the files to multiple locations so there is a delay while it does so. There is also some overhead from NFS, as each filesystem operation is a network call. What EFS is designed for is actually parallel operations. But rsync or cp can’t run in parallel, so you’ll need to manually batch up your files or use this tool that was referenced in the document above called fpsync (Filesystem partitioner sync).

What fpsync can do is split a directory of files up into chunks, and then send those contents in parallel via rsync. This is also possible with GNU Parallel but you’d have to write your own script. fpsync was available on CentOS, and probably many other distributions. Let’s run it out of the box:

fpsync /big_dir /efs_dir

real    0m59.790s
user    0m1.975s
sys     0m3.925s

Not much of an improvement…but why? Because fpsync doesn’t run in parallel by default, and you have to tweak it a bit. Let’s process 100 files at a time using 10 concurrent runners:

# time fpsync -f 100 -n 10 -v /big_dir /efs_dir
1662569967 Info: Run ID: 1662569967-43986
1662569967 ===> Analyzing filesystem...
1662569968 <=== Fpart crawling finished
1662569980 <=== Parts done: 29/29 (100%), remaining: 0
1662569980 <=== Time elapsed: 13s, remaining: ~0s (~0s/job)
1662569980 <=== Fpsync completed without error in 13s.

real    0m13.467s
user    0m2.086s
sys     0m4.400s

Much better! But let’s try more concurrent runners. Since we had 3000 files, there would have been a queue in our last command (100*10 = 1000). So let’s run 50 batches of 50 files each:

# fpsync -f 50 -n 50 -v /big_dir /efs_dir
1662570120 Info: Run ID: 1662570120-51913
1662570120 ===> Analyzing filesystem...
1662570122 <=== Fpart crawling finished
1662570129 <=== Parts done: 58/58 (100%), remaining: 0
1662570129 <=== Time elapsed: 9s, remaining: ~0s (~0s/job)
1662570129 <=== Fpsync completed without error in 9s.

real    0m8.903s
user    0m2.093s
sys     0m4.868s

So, the more concurrent copy operations we can run, the better.

On a regular disk this wouldn’t have any effect since the filesystem operations are negligible and your only bottleneck is the disk speed. It might even slow it down. There may be some other options inside of fpsync that would speed it up even more. What about rsync --inplace? This would eliminate a step that rsync would usually take, which is to create a new file, then rename it.

#  time fpsync -o "--inplace" -f 100 -n 50 -v /big_dir /efs_dir
1662584365 Info: Run ID: 1662584365-126224
1662584365 ===> Analyzing filesystem...
1662584367 <=== Fpart crawling finished
1662584371 <=== Parts done: 29/29 (100%), remaining: 0
1662584371 <=== Time elapsed: 6s, remaining: ~0s (~0s/job)
1662584371 <=== Fpsync completed without error in 6s.

real    0m5.872s
user    0m1.634s
sys     0m3.438s

Running batches of 100 brought it down to under 6s. After that it started to get slower. Also running a huge number of rsyncs and small batches got slower. This is likely due to the system itself – after all, it’s running 250+ instance of rsync.

replacing FX-101B board in a Haier HVTEC12DABS

The Haier HVTEC12DABS is a 12-bottle wine fridge with a 2-zone thermoelectric (peltier) cooling setup.

My fridge started acting up. Beeping on start, front panel dead, no cooling. From what I read in a very useful thread at All About Circuits it’s possible to replace commonly blown capacitors, fuses, and transistors. But I am not that handy and I’d probably mess something else up in the process.

So I looked for replacement boards. This unit has two FX-101B boards, manufactured by “Foshan Hanyi Computer”:

The board is labelled as “FX-101B 10.5V PCB120606F1”. There is another marking “SH14611” with a date of “12.07.12”.

I wanted to replace both boards but the FX-101B is impossible to find. The “B” variant must have been made specifically for Haier, as other wine fridges have an FX-101 or FX-102 board. The differences I found:

  • the TEC (thermo-electric cooler) plug is a spade terminal rather than pins
  • there is an additional NTC (thermristor) connector for temperature control

I found that they sell these boards on Aliexpress in all different varieties (search for “FX-101 board”). But I could not find an FX-101B board, only the FX-101 and FX-102 which is apparently compatible. So I took a shot – $72.10 shipped for two FX-101 boards, from China. Now I’m thinking the FX-102 might have been a new board with better components but I didn’t really know much at the time and just went with the part I thought was the most compatible.

Note: Stock image because I forgot to take a picture before I put them in. When they arrived, they were marked 110v and 10.5v as requested. I found out later that if you find a 110v 12.5v board, that should work as well. 220v will obviously not work if your previous board is 110v and vice versa. Notice the different connector for the TEC power and the new NTC connector in the upper right.

First I had to cut off the existing TEC connector in the fridge and crimp spade disconnects to them which was a $4 purchase from Home Depot. Note: the FAN1 and FAN2 are different! One is hot side and one is cool side. The cool side (inside) should always spin, the hot side (outside) will only spin if the board is cooling. Switch your fan connectors if this happens. I plugged everything back in and started the wine cooler up.

Joy! The front panel was no longer blinking and chirping and it looked like it was cooling. But alas, the red lights on the back of the boards were lit up. This means that the unit has power but has not been requested to cool. The cool side (inside) fans were spinning. The hot side (outside) fans were not spinning. Multimeter showed no power to the TEC. I figured this had something to do with the new NTC connector and that maybe the board was expecting to be turned on via a thermristor instead of the control from the front panel.

I used a jumper to short the NTC pins together and the cooler came alive. The green LED lit up and the back fans started spinning. It then cooled down the set temperature.

I confirmed against my suspicions, even with the NTC pins jumped, the front panel temperature was still controlling cooling activation. I had thought that this would have made the fridge cool infinitely as it thought it was too warm (a thermristor will increase resistance as it gets hotter). Why this is, I don’t know. I guess the remote control overrides the thermristor, or, if it detects very low resistance it assumes there is no thermristor. Now the fridge cools to the requested temperature and turns off when not needed. Working like brand new!

Attaching Rules conditions to a config entity

Enhancing modules with Rules-based conditions was very easy in D7. Using hook_default_rules_configuration we could dynamically generate a bunch of rules called mymodule_rule_[some_key], use rules_ui()->config_menu() to add the menu items for the Rules admin UI, then invoke the generated components to evaluate conditions. Every entity or option would have its own Rules component that we can edit and add arbitrary conditions. Some examples of this in D7 were:

  • Payment methods (Ubercart/Commerce)
  • Coupons
  • Tax rules
  • Block visibility
  • User access or eligibility

And anything where you could not possibly know of the conditions that would be needed. Some of the above were changed to use Core conditions in D8, but that didn’t cut it for our use case since I could not possibly write a new condition for every requirement that came up. Real life examples of these are:

  • A user can only claim a certain kind of course credit when the credit code on the course contains specific characters and the user is from Florida.
  • The user can only use the payment method when there is a valid role attached to the user and specific products are in the cart.
  • A user is not eligible to receive a certain type of credit when they are eligible to receive a certain type of credit.
  • A quiz taker can only see correct answers once two weeks have passed and the user exhausted two attempts.

These aren’t out of the ordinary and we would be writing custom PHP if/else trees every day. For a SaaS-like product this is not ideal.

It’s a little trickier to add arbitrary conditions to entities but well worth it in the end. Rules provides a test module that you can look at: rules_test_ui_embed. This example illustrates using 1 rule component embedded into a page. But we need to build Rules into all instances of a configuration entity.

Rules provides an interface RulesUiComponentProviderInterface that we can use to store component configuration on our entity types. This was added in https://www.drupal.org/project/rules/issues/2659016 and https://www.drupal.org/project/rules/issues/2671056 but so far, there don’t seem to be any contributed modules that implement this! Rules does use it for its own action and condition components.

There is documentation for extending Rules with new conditions and actions, but it is pretty lacking around integration. There is some embedded developer documentation so let’s take a look.

If we look at rules_test_ui_embed we see that there is some sort of plugin file – rules_test_ui_embed.rules_ui.yml. That must define something!

rules_test_ui_embed.rules.ui.yml:

rules_test_ui_embed.settings_conditions:
   label: 'Conditions embedded in Rules-test-UI-embed module settings'
   base_route: rules_test_ui_embed.settings
   settings:
     config_name: rules_test_ui_embed.settings
     config_key: conditions

The above defines a Rules UI plugin which will create routes on rules_test_ui_embed.settings .The configuration for the Rules component will be saved to rules_test_ui_embed.settings under the conditions key. But that doesn’t work for us, we need to have multiple components on multiple entities.

There’s another parameter in RulesUiConfigHandler we can use to allow wildcard editing of components: config_parameter

It appears that config_parameter and config_key can be used to dynamically set which configuration object and key will be updated. With a little trial and error I applied it to Quiz feedback types. Feedback types hold sets of review options that display feedback to quiz takers after they answer a question or finish an entire quiz. They can also be used for post-review feedback, in the case of revisiting the quiz after 2 weeks. Only seeing correct answers after 3 attempts, only seeing instructor feedback once given a role, etc…

Let’s assume that we already have a QuizFeedbackType entity to allow creation of custom feedback “times”, and all the edit forms are already set up. We want to add conditions to each feedback type so that we can conditionally display their items. In Quiz we have two built in: “Question” and “End”.

Define route and *.rules_ui.yml

This will indicate that we want Rules UI functionality appended to a route that we will also create. It will also tell Rules that we want the component to be saved onto the object loaded from the quiz_feedback_type parameter. Note how _rules_ui option on the route matches the plugin name defined in quiz_rules.ui.yml:

quiz.rules.ui.yml:

quiz_feedback.rules_ui_conditions:
  label: 'Embedded quiz feedback conditions'
  base_route: entity.quiz_feedback_type.conditions
  settings:
    config_key: component
    config_parameter: quiz_feedback_type

quiz.routing.yml:

[...]
entity.quiz_feedback_type.conditions:
  path: 'admin/quiz/feedback/type/{quiz_feedback_type}/conditions'
  defaults:
    _form: '\Drupal\quiz\Form\QuizFeedbackConditionsForm'
    _title: 'Feedback conditions'
  requirements:
    _permission: 'administer quiz'
  options:
    _rules_ui: quiz_feedback.rules_ui_conditions
    parameters:
      quiz_feedback_type:
        type: 'entity:quiz_feedback_type'

Define new form for editing a component

This is a normal form that extends ConfigFormBase, but is provided with a Rules UI handler from the plugin definition that matches the route above. Most of this code is copied from rules_test_ui_embed:

QuizFeedbackConditionsForm.php

[...]
class QuizFeedbackConditionsForm extends ConfigFormBase {

  /**
   * The RulesUI handler of the currently active UI.
   *
   * @var RulesUiConfigHandler
   */
  protected $rulesUiHandler;

  /**
   * {@inheritdoc}
   */
  protected function getEditableConfigNames() {
    return [];
  }

  /**
   * {@inheritdoc}
   */
  public function getFormId() {
    return 'quiz_feedback_conditions';
  }

  /**
   * {@inheritdoc}
   */
  public function buildForm(array $form, FormStateInterface $form_state, RulesUiConfigHandler $rules_ui_handler = NULL) {
    $form = parent::buildForm($form, $form_state);
    $this->rulesUiHandler = $rules_ui_handler;

    $form['conditions'] = $this->rulesUiHandler->getForm()
      ->buildForm([], $form_state);

    $form['actions']['cancel'] = [
      '#type' => 'submit',
      '#limit_validation_errors' => [['locked']],
      '#value' => $this->t('Cancel'),
      '#submit' => ['::cancel'],
    ];

    return $form;
  }

  /**
   * {@inheritdoc}
   */
  public function validateForm(array &$form, FormStateInterface $form_state) {
    parent::validateForm($form, $form_state);
    $this->rulesUiHandler->getForm()
      ->validateForm($form['conditions'], $form_state);
  }

  /**
   * {@inheritdoc}
   */
  public function submitForm(array &$form, FormStateInterface $form_state) {
    $this->rulesUiHandler->getForm()
      ->submitForm($form['conditions'], $form_state);

    // Save the configuration that submitForm() updated (the config entity).
    $config = $this->rulesUiHandler->getConfig();
    $config->save();

    // Also remove the temporarily stored component, it has been persisted now.
    $this->rulesUiHandler->clearTemporaryStorage();

    parent::submitForm($form, $form_state);
  }

  /**
   * Form submission handler for the 'cancel' action.
   */
  public function cancel(array $form, FormStateInterface $form_state) {
    $this->rulesUiHandler->clearTemporaryStorage();
    $this->messenger()->addMessage($this->t('Canceled.'));
    $form_state->setRedirectUrl($this->rulesUiHandler->getBaseRouteUrl());
  }

}

In buildForm we take in the Rules UI handler and use it to generate the condition form. In submitForm, the Rules UI handler will notify our Rules component “provider” that there is a component that has to be saved.

Implement RulesUiComponentProviderInterface

The rulesUiHandler from above requires the entity type to handle getting the Rules component and saving it onto itself since we are not specifying a static config_name or config_key We add the component property to config_export, then we implement RulesUiComponentProviderInterface and implement the 2 methods:

/**
 * @ConfigEntityType(
[...]
 *   config_export = {
 *     "id",
 *     "label",
 *     "component"
 *   },
 */
class QuizFeedbackType extends ConfigEntityBase implements RulesUiComponentProviderInterface {

  /**
   * {@inheritdoc}
   */
  public function getComponent() {
    if (empty($this->component)) {
      // Provide a default for now.
      $this->component = [
        'expression' => ['id' => 'rules_and'],
        'context_definitions' => [
          'quiz_result_answer' => [
            'type' => 'entity:quiz_result',
            'label' => 'Quiz result',
            'description' => 'Quiz result to evaluate feedback',
          ],
        ],
      ];
    }

    if (!isset($this->componentObject)) {
      $this->componentObject = RulesComponent::createFromConfiguration($this->component);
    }
    return $this->componentObject;
  }

  /**
   * {@inheritdoc}
   */
  public function updateFromComponent(RulesComponent $component) {
    $this->component = $component->getConfiguration();
    $this->componentObject = $component;

    return $this;
  }

}
  • In getComponent() we check to see if the entity already has conditions, and return a RulesComponent. If it does not, we provide a default that intakes a QuizResult entity to evaluate.
  • In updateFromComponent(), we get the RulesComponent and store it on the entity.

And there it goes, components being attached to entities.

Now that the components are stored on an entity that implements RulesUiComponentProviderInterface, we can invoke the rule in our code to validate the conditions:

/* @var $component RulesComponent */
$component = QuizFeedbackType::load('question')->getComponent();

// Answer 31 has not been answered.
$not_finished = QuizResult::load(31);
$component->setContextValue('quiz_result', $not_finished);
$x = $component->getExpression()->executeWithState($component->getState());
var_dump($x); // $x = FALSE

// Answer 32 has been answered, show feedback.
$finished = QuizResult::load(32);
$component->setContextValue('quiz_result', $finished);
$y = $component->getExpression()->executeWithState($component->getState());
var_dump($y); // $y = TRUE

Reference: https://www.drupal.org/project/rules/issues/3117749

replacing the haproxy error page in openshift origin

first, create a configmap secret which will contain the 503 page

apiVersion: v1
data:
  error-page-503.http: |-
    HTTP/1.0 503 Service Unavailable
    Pragma: no-cache
    Cache-Control: private, max-age=0, no-cache, no-store
    Connection: close
    Content-Type: text/html

    <html><head><body>We'll be back!</body></head></html>
kind: ConfigMap
metadata:
  name: haproxy-error-page
  namespace: default

then create it

oc -n default create -f haproxy-error-page.yml

now you need to add it as a volume

oc -n default set volume --add --configmap-name=haproxy-error-page dc/router -m /var/lib/haproxy/conf/error-page-503.http --name=haproxy-config-volume

and patch the router config (see https://github.com/openshift/origin/issues/16275 for why this is necessary)

oc -n default patch dc router -p '{"spec":{"template":{"spec":{"containers":[{"volumeMounts":[{"mountPath":"/var/lib/haproxy/conf/error-page-503.http","subPath":"error-page-503.http","name":"haproxy-config-volume"}],"name":"router"}]}}}}

selinux also has to be disabled on the router box (for now)